diff --git a/projekt/README.md b/projekt/README.md
index 1b9156f..9bcbd9b 100644
--- a/projekt/README.md
+++ b/projekt/README.md
@@ -96,4 +96,24 @@ Dzięki głowie modelu, która jest przeznaczona specjalnie do zadania klasyfika
 ### T5 oraz FLAN-T5
 Modele T5 oraz FLAN-T5 to modele text-2-text, których głównym celem jest multizadaniowość. Nie są zoptymalizowane do zadania klasyfikacji, ale mimo to osiągnęły dobry wynik. W modelu T5 bardzo ważne jest określnie zadania w prefixie inputu, które będzie wykonywał model. Testowaliśmy różne kombinacje - brak prefixu, dokładny opis zadania, różne klasy wyjściowe, ale najlepsze rezultaty uzyskał prefix 'binary classification: ' z klasami wyjściowymi '0' lub '1'. Niski MCC świadczy o niezbalansowaniu predykcji.
 <br/><br/>
-FLAN-T5 to zoptymalizowany model T5. Został dodatkowo finetunowany na większym zbiorze danych. Przy uczeniu FLAN-T5 zastosowaliśmy metodę few-shot learning. Model mimo bardzo małej ilości przykładów osiągnął accuracy na poziomie 46%, jednak wynik MCC wskazuje na duże niezbalansowanie i losowe predykcje. Jest to jednak bardzo ciekawe, że wykorzystując tylko 4 przykady do uczenia, model może odpowiadać z pasującymi wynikami (niekoniecznie prawdziwymi).
\ No newline at end of file
+FLAN-T5 to zoptymalizowany model T5. Został dodatkowo finetunowany na większym zbiorze danych. Przy uczeniu FLAN-T5 zastosowaliśmy metodę few-shot learning. Model mimo bardzo małej ilości przykładów osiągnął accuracy na poziomie 46%, jednak wynik MCC wskazuje na duże niezbalansowanie i losowe predykcje. Jest to jednak bardzo ciekawe, że wykorzystując tylko 4 przykady do uczenia, model może odpowiadać z pasującymi wynikami (niekoniecznie prawdziwymi).
+
+### FLAN-T5 - testy
+|Wersja|Rozmiar|Prefiks|Acc|
+|---|---|---|---|
+|1|base|SMS - Content of the text message: <br />Result - Answer if this text message is spam or not|0.503593244699964|
+|2|large|SMS - Content of the text message: <br />Result - Answer if this text message is spam or not|0.46119295724038806|
+|3|base|SMS - The text message contains the following content: <br />Result - Is it true that the content of a text message is spam:|0.2551203736974488|
+|4|large|SMS - The text message contains the following content: <br />Result - Is it true that the content of a text message is spam:|0.1433704635285663|
+|5|base|SMS - The text message: <br />Result - Is this sms spam? Ans:|0.28404599353215954|
+|6|large|SMS - The text message: <br />Result - Is this sms spam? Ans:|0.20786920589292132|
+
+
+### Modele FLAN_T5
+
+1. https://drive.google.com/drive/folders/1XO2TEMIKKFXCwpeW1X51hpC89LShCZ6c?usp=share_link
+2. https://drive.google.com/drive/folders/1-3DEVCcxhRGSmEVx3jK4SUVVIUixNKqS?usp=share_link
+3. https://drive.google.com/drive/folders/1-0Ct4JFzRhyo3bGuOc9ttZhaV4ghQRFx?usp=share_link
+4. https://drive.google.com/drive/folders/1-1YgvI9-2cpkRO1hSlYU-hJFH2a1rmDp?usp=share_link
+5. https://drive.google.com/drive/folders/1-4KINMK9L_VyCZGpCDueazmF2DUbRXTn?usp=share_link
+6. https://drive.google.com/drive/folders/1-3gQuzZVq6IMqcY1nDu2hazVD0dN6F-K?usp=share_link
\ No newline at end of file
diff --git a/projekt/Ver1_FLAN-T5_sms_spam.ipynb b/projekt/Ver1_FLAN-T5_sms_spam.ipynb
new file mode 100644
index 0000000..e33d2ac
--- /dev/null
+++ b/projekt/Ver1_FLAN-T5_sms_spam.ipynb
@@ -0,0 +1 @@
+{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"},"gpuClass":"standard","widgets":{"application/vnd.jupyter.widget-state+json":{"ba9d40cd07c34f4fb1f9fbd73092fbf6":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_9de313fab7ae49b080fb5192acb456c0","IPY_MODEL_211c322124ff451c8d6cddbc842e650b","IPY_MODEL_9e285551432c4b5daa6afa6049a0a88d"],"layout":"IPY_MODEL_e355e0d5122a46d785cdc07117a66990"}},"9de313fab7ae49b080fb5192acb456c0":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_2db46abba29f4f9789de6fdbec1af7a0","placeholder":"​","style":"IPY_MODEL_9bff66fdf5c94944a63c5525685a71af","value":"Downloading builder script: 100%"}},"211c322124ff451c8d6cddbc842e650b":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_27f2295f6b0c4e2f97f511514901b60d","max":3208,"min":0,"orientation":"horizontal","style":"IPY_MODEL_24dd9a57eb984bdaa61c77601a0f5052","value":3208}},"9e285551432c4b5daa6afa6049a0a88d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_752ab7a625c44966b4d0204355ed1d03","placeholder":"​","style":"IPY_MODEL_e575215f324e4f959cee7086a32c4039","value":" 3.21k/3.21k [00:00&lt;00:00, 170kB/s]"}},"e355e0d5122a46d785cdc07117a66990":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2db46abba29f4f9789de6fdbec1af7a0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9bff66fdf5c94944a63c5525685a71af":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"27f2295f6b0c4e2f97f511514901b60d":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"24dd9a57eb984bdaa61c77601a0f5052":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"752ab7a625c44966b4d0204355ed1d03":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e575215f324e4f959cee7086a32c4039":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7ae5fccd7dc442f0b6edf652210e8f45":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_a976451070bc43d3a5f2504e6b2441b6","IPY_MODEL_842cb52b1c214db9a5704752497d5944","IPY_MODEL_20dd392ebbbf4027827a722f1b5244de"],"layout":"IPY_MODEL_b2f23e1760284b5f8b6428d499556825"}},"a976451070bc43d3a5f2504e6b2441b6":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_dc6f0fbc82a84b6f9e01e085a66260b9","placeholder":"​","style":"IPY_MODEL_36540af07eb34debbc483e635a1aa978","value":"Downloading metadata: 100%"}},"842cb52b1c214db9a5704752497d5944":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_67a513749d4d4e6f8fce80debaafe1a9","max":1687,"min":0,"orientation":"horizontal","style":"IPY_MODEL_ab6920be0d57483db97d91d30b92bc0a","value":1687}},"20dd392ebbbf4027827a722f1b5244de":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b3cd7dbebf6b488a8a4d7146e0dc89ec","placeholder":"​","style":"IPY_MODEL_620ed6142c3e4a96b49ce51dca45dbb4","value":" 1.69k/1.69k [00:00&lt;00:00, 86.7kB/s]"}},"b2f23e1760284b5f8b6428d499556825":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"dc6f0fbc82a84b6f9e01e085a66260b9":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"36540af07eb34debbc483e635a1aa978":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"67a513749d4d4e6f8fce80debaafe1a9":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ab6920be0d57483db97d91d30b92bc0a":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"b3cd7dbebf6b488a8a4d7146e0dc89ec":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"620ed6142c3e4a96b49ce51dca45dbb4":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c4ba45f6ee0e4edb9cb1ee26fadeb01a":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_f87e37e286a74e94adc773e74c2c09c9","IPY_MODEL_c6ab86832ff94f818329bd34d6e6efe2","IPY_MODEL_4c8fbd675bd24a279348316ad06b6676"],"layout":"IPY_MODEL_abb70e15da2849edbc926078009f19de"}},"f87e37e286a74e94adc773e74c2c09c9":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e147057451f844de84f8c9eb7896550f","placeholder":"​","style":"IPY_MODEL_d9fd300cff5a4b5d97cddcb217677465","value":"Downloading readme: 100%"}},"c6ab86832ff94f818329bd34d6e6efe2":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_5e95b5a9e0724361a84b729b547ca3ad","max":4872,"min":0,"orientation":"horizontal","style":"IPY_MODEL_148d65aedcb14ae59b2b79363ed919f7","value":4872}},"4c8fbd675bd24a279348316ad06b6676":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_37e3971106aa456981303debb9b4c225","placeholder":"​","style":"IPY_MODEL_0d66ab66ba6744f99ec4c63334eeb106","value":" 4.87k/4.87k [00:00&lt;00:00, 174kB/s]"}},"abb70e15da2849edbc926078009f19de":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e147057451f844de84f8c9eb7896550f":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d9fd300cff5a4b5d97cddcb217677465":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5e95b5a9e0724361a84b729b547ca3ad":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"148d65aedcb14ae59b2b79363ed919f7":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"37e3971106aa456981303debb9b4c225":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0d66ab66ba6744f99ec4c63334eeb106":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"0ef3d6b2ea5542eea95c2a8058652cde":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_834d071dc2844826afe2cba68d7b7ee0","IPY_MODEL_ddc9057d5fcb4f06ad92e89181850c2b","IPY_MODEL_3d6f5d5597f3442ca8d50e82c37ea8f0"],"layout":"IPY_MODEL_7c0f4fac68fc49f8b288d8348196ce73"}},"834d071dc2844826afe2cba68d7b7ee0":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7317d2612bf54f849071aa6c2846d2a0","placeholder":"​","style":"IPY_MODEL_a85008da935b414dba2c9cec5a81d998","value":"Downloading data: 100%"}},"ddc9057d5fcb4f06ad92e89181850c2b":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_7c15d362b51c4466a9d994208a1058b7","max":203415,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1e53fe4a87fc4ce3bdf2145fb1f1db7a","value":203415}},"3d6f5d5597f3442ca8d50e82c37ea8f0":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_328ff44536394df59f5aad906b4ba472","placeholder":"​","style":"IPY_MODEL_0abafc781cd14f36af17144f84b4e0ee","value":" 203k/203k [00:00&lt;00:00, 783kB/s]"}},"7c0f4fac68fc49f8b288d8348196ce73":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7317d2612bf54f849071aa6c2846d2a0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a85008da935b414dba2c9cec5a81d998":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7c15d362b51c4466a9d994208a1058b7":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1e53fe4a87fc4ce3bdf2145fb1f1db7a":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"328ff44536394df59f5aad906b4ba472":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0abafc781cd14f36af17144f84b4e0ee":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b0e9810c15bf4877b12cbe5b2b391c11":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b11ff7e8103d4f19808023ac61f1c495","IPY_MODEL_a1e2e0b7b94b4c29b10e256d4665ac1c","IPY_MODEL_f6308b3e42ee4759982e054f7f1fc13b"],"layout":"IPY_MODEL_a525b423a38446e7acd4adf0806ce16b"}},"b11ff7e8103d4f19808023ac61f1c495":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d777deca350c43659ae1c9de6809f975","placeholder":"​","style":"IPY_MODEL_5b874a9c118b4aba94fb8df333829703","value":"Generating train split:  67%"}},"a1e2e0b7b94b4c29b10e256d4665ac1c":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"","description_tooltip":null,"layout":"IPY_MODEL_d907fcfb69d44b9585c46df7b97980cc","max":5574,"min":0,"orientation":"horizontal","style":"IPY_MODEL_6b0be17848444747b51137f20854ad53","value":5574}},"f6308b3e42ee4759982e054f7f1fc13b":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_51f48056fa734f969b98d6779c75fda4","placeholder":"​","style":"IPY_MODEL_1a7a53fd227441e7bdcd31050b189a85","value":" 3733/5574 [00:00&lt;00:00, 17820.10 examples/s]"}},"a525b423a38446e7acd4adf0806ce16b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":"hidden","width":null}},"d777deca350c43659ae1c9de6809f975":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5b874a9c118b4aba94fb8df333829703":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d907fcfb69d44b9585c46df7b97980cc":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6b0be17848444747b51137f20854ad53":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"51f48056fa734f969b98d6779c75fda4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1a7a53fd227441e7bdcd31050b189a85":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"79b64a59aa904f46803c0013bfaa67c7":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8c09a31ddcd94408baea9bb143b3608f","IPY_MODEL_2af0dd8e534d4e25a89ac415d3f7a536","IPY_MODEL_114db7aaead5484ba5379731f0f74de5"],"layout":"IPY_MODEL_1f293d6ea070431e90f8ad1a79c42822"}},"8c09a31ddcd94408baea9bb143b3608f":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_aed3d46ba465417cab5d75d09d395d54","placeholder":"​","style":"IPY_MODEL_5246a27a3ea94e9cb6d1baafe4a51cdd","value":"100%"}},"2af0dd8e534d4e25a89ac415d3f7a536":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_2d5a8b6f1e70487781a88da6dc8b6aa5","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_90b9696a21794d99b9d4dcd34ec7866e","value":1}},"114db7aaead5484ba5379731f0f74de5":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f1221b8b45f14027855dc31cbc932adc","placeholder":"​","style":"IPY_MODEL_1836db5096354e18810611ffd14c39b8","value":" 1/1 [00:00&lt;00:00, 42.94it/s]"}},"1f293d6ea070431e90f8ad1a79c42822":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"aed3d46ba465417cab5d75d09d395d54":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5246a27a3ea94e9cb6d1baafe4a51cdd":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2d5a8b6f1e70487781a88da6dc8b6aa5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"90b9696a21794d99b9d4dcd34ec7866e":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"f1221b8b45f14027855dc31cbc932adc":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1836db5096354e18810611ffd14c39b8":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"130f4a8f140d43e9993f6f158d643f91":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_052168c2faa04049bec6605ec9fc654a","IPY_MODEL_8baade75913f4bc28033f916db729edd","IPY_MODEL_9032dbaf99f04ef8a410969b7345928f"],"layout":"IPY_MODEL_d703073401d84414b90c3d71f9914e39"}},"052168c2faa04049bec6605ec9fc654a":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6178481579ef4b2086d730108c77132e","placeholder":"​","style":"IPY_MODEL_81761b679c9f462a907ee227ca4e0e3d","value":"Downloading (…)okenizer_config.json: 100%"}},"8baade75913f4bc28033f916db729edd":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_cad9c06510c44d76b8873ab1f1b9e0ce","max":2537,"min":0,"orientation":"horizontal","style":"IPY_MODEL_da24d4f940734731980bbf4511b7f388","value":2537}},"9032dbaf99f04ef8a410969b7345928f":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0137abcc3a8c4a7fbc77dfc578f8bd28","placeholder":"​","style":"IPY_MODEL_47340075461a44d981ecea3883273a3c","value":" 2.54k/2.54k [00:00&lt;00:00, 91.2kB/s]"}},"d703073401d84414b90c3d71f9914e39":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6178481579ef4b2086d730108c77132e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"81761b679c9f462a907ee227ca4e0e3d":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"cad9c06510c44d76b8873ab1f1b9e0ce":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"da24d4f940734731980bbf4511b7f388":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"0137abcc3a8c4a7fbc77dfc578f8bd28":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"47340075461a44d981ecea3883273a3c":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"09f475e7464c4101a94dce60d034ac12":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_5244aabb420f4970954781124f07ee1c","IPY_MODEL_c477c1291af84638a961dcf1d43a0eb5","IPY_MODEL_9dd0b870a8da4380af8e15c6e4f02dae"],"layout":"IPY_MODEL_d1c3f34e830841c39e7c86f754843903"}},"5244aabb420f4970954781124f07ee1c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e42f6346d52346779e2ec76061abf4db","placeholder":"​","style":"IPY_MODEL_6a53778d0eb041959e476ead69b761ad","value":"Downloading (…)&quot;spiece.model&quot;;: 100%"}},"c477c1291af84638a961dcf1d43a0eb5":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_4831844601e548849e3c3ae6c5c0f734","max":791656,"min":0,"orientation":"horizontal","style":"IPY_MODEL_9ad9b94a8d5b41a99fae31fac4782dae","value":791656}},"9dd0b870a8da4380af8e15c6e4f02dae":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_1a8502e4909b493e8a0015c9406250a2","placeholder":"​","style":"IPY_MODEL_faed3ee84d1649f6aee33545623c3414","value":" 792k/792k [00:00&lt;00:00, 5.79MB/s]"}},"d1c3f34e830841c39e7c86f754843903":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e42f6346d52346779e2ec76061abf4db":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6a53778d0eb041959e476ead69b761ad":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4831844601e548849e3c3ae6c5c0f734":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9ad9b94a8d5b41a99fae31fac4782dae":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1a8502e4909b493e8a0015c9406250a2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"faed3ee84d1649f6aee33545623c3414":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"bf8558287dc54fe19fd4eca5043dc182":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_bb3022078d654d90aad064587d5fcb81","IPY_MODEL_79d8d0f473d54e518023d13b4896dcb0","IPY_MODEL_1cdf0415e5234288a5ed6900a61bceb1"],"layout":"IPY_MODEL_ef9cd89580534dea85fcd3ef8e129d37"}},"bb3022078d654d90aad064587d5fcb81":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0bd704ed93f249a882ea09a9b4760856","placeholder":"​","style":"IPY_MODEL_bb343e8421494468ba70a338e46ce97b","value":"Downloading (…)/main/tokenizer.json: 100%"}},"79d8d0f473d54e518023d13b4896dcb0":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_b5a1ca49058e4988a6a8f4f072bd8bb6","max":2424064,"min":0,"orientation":"horizontal","style":"IPY_MODEL_360ee70e58784c4ab7a08333afd4ff30","value":2424064}},"1cdf0415e5234288a5ed6900a61bceb1":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e51e0982800a4dc3bea1feeab7e3b5f1","placeholder":"​","style":"IPY_MODEL_72f4d1baccca4021b6392763c3f6068d","value":" 2.42M/2.42M [00:00&lt;00:00, 17.7MB/s]"}},"ef9cd89580534dea85fcd3ef8e129d37":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0bd704ed93f249a882ea09a9b4760856":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bb343e8421494468ba70a338e46ce97b":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b5a1ca49058e4988a6a8f4f072bd8bb6":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"360ee70e58784c4ab7a08333afd4ff30":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"e51e0982800a4dc3bea1feeab7e3b5f1":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"72f4d1baccca4021b6392763c3f6068d":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"453344c1de7c4fef9ad94d31fd1c4972":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_afa1d46f350f439ea7371b29b11f12ee","IPY_MODEL_29ce5457214f4b7cbac8a866ec805fa0","IPY_MODEL_c9f71fbae55448d69d13216a4a88f040"],"layout":"IPY_MODEL_fc0695782e4f4ddc8c3b9140bef9a482"}},"afa1d46f350f439ea7371b29b11f12ee":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0bf540f48dba48eaa2ef411032e8372c","placeholder":"​","style":"IPY_MODEL_bf7f88a03d1e4bb68ca7e3df611525ba","value":"Downloading (…)cial_tokens_map.json: 100%"}},"29ce5457214f4b7cbac8a866ec805fa0":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_8d82e208f6b44e409050891ab76f1551","max":2201,"min":0,"orientation":"horizontal","style":"IPY_MODEL_2ba133a583514a528fae49101dddd523","value":2201}},"c9f71fbae55448d69d13216a4a88f040":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6e1f83ef1afc42e0a0d8562c1f871067","placeholder":"​","style":"IPY_MODEL_c0194dbf3d5842fa8c1437def7113be6","value":" 2.20k/2.20k [00:00&lt;00:00, 99.0kB/s]"}},"fc0695782e4f4ddc8c3b9140bef9a482":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0bf540f48dba48eaa2ef411032e8372c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bf7f88a03d1e4bb68ca7e3df611525ba":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8d82e208f6b44e409050891ab76f1551":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2ba133a583514a528fae49101dddd523":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"6e1f83ef1afc42e0a0d8562c1f871067":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c0194dbf3d5842fa8c1437def7113be6":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8b2028bc0dd34cd49ff063356924f2d1":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b61f9f83affd4bffa20e35550e1dcf11","IPY_MODEL_9402536c51f54eebb6f9d6ebff692ed3","IPY_MODEL_27d09ed90c3048ef92deab33ed7b1b5c"],"layout":"IPY_MODEL_fda0aada96374e159ddfad0831e795d5"}},"b61f9f83affd4bffa20e35550e1dcf11":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_81ea8a9d70a8416db4d60b4596e00f92","placeholder":"​","style":"IPY_MODEL_c2f2cf8567514612a788b4ade694daba","value":"Downloading (…)lve/main/config.json: 100%"}},"9402536c51f54eebb6f9d6ebff692ed3":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a771d23698b641e09bbfc51d004a7358","max":1404,"min":0,"orientation":"horizontal","style":"IPY_MODEL_28ae83bc70ba41d7aad0f785d79e6600","value":1404}},"27d09ed90c3048ef92deab33ed7b1b5c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_032276c9596f4100965fb757c5741709","placeholder":"​","style":"IPY_MODEL_04be2018587746949e3def7c3a51d7a9","value":" 1.40k/1.40k [00:00&lt;00:00, 48.7kB/s]"}},"fda0aada96374e159ddfad0831e795d5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"81ea8a9d70a8416db4d60b4596e00f92":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c2f2cf8567514612a788b4ade694daba":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a771d23698b641e09bbfc51d004a7358":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"28ae83bc70ba41d7aad0f785d79e6600":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"032276c9596f4100965fb757c5741709":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"04be2018587746949e3def7c3a51d7a9":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"62e2e7c6fc834eca82be7f85c9fa9701":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_060b98b462b84394b4aa833f31abcc69","IPY_MODEL_e572c1b7e70b4abc97e390ad78ac6f3f","IPY_MODEL_3eb033bb16ce4534a7f7538c740c1d39"],"layout":"IPY_MODEL_a38561dd1f81429d9292affd976070e0"}},"060b98b462b84394b4aa833f31abcc69":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_2097b347701f4eda889728f50f62252f","placeholder":"​","style":"IPY_MODEL_3276c774bee745a791d14b3d7490cc8c","value":"Downloading (…)&quot;pytorch_model.bin&quot;;: 100%"}},"e572c1b7e70b4abc97e390ad78ac6f3f":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a47dba0fd1f04505927b837d252fe5fe","max":990402637,"min":0,"orientation":"horizontal","style":"IPY_MODEL_b4e93db0bd734f9ebccc281be866fede","value":990402637}},"3eb033bb16ce4534a7f7538c740c1d39":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6a8020dba5c94c69a4c39abc890b877e","placeholder":"​","style":"IPY_MODEL_20edf60841f54c5b9b5863a20a9ea0a5","value":" 990M/990M [00:10&lt;00:00, 95.9MB/s]"}},"a38561dd1f81429d9292affd976070e0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2097b347701f4eda889728f50f62252f":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3276c774bee745a791d14b3d7490cc8c":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a47dba0fd1f04505927b837d252fe5fe":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b4e93db0bd734f9ebccc281be866fede":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"6a8020dba5c94c69a4c39abc890b877e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"20edf60841f54c5b9b5863a20a9ea0a5":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6eb421544bdd4100b92e2dc12e783485":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_c3ee8e64d4024205896cb81a77d08733","IPY_MODEL_29d772f88bc3404585377162d5762272","IPY_MODEL_f7de2c8b9f1a4baf87a31cbadeb6424f"],"layout":"IPY_MODEL_a8ea02156db1495999105c450d4172bf"}},"c3ee8e64d4024205896cb81a77d08733":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d6136a686f1f4b7da80d48e12d0bc79c","placeholder":"​","style":"IPY_MODEL_a8e477092733491f9ece5528a272652d","value":"Downloading (…)neration_config.json: 100%"}},"29d772f88bc3404585377162d5762272":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a4f1cf289630449999dc9c279ad069aa","max":147,"min":0,"orientation":"horizontal","style":"IPY_MODEL_2aa24b5ff19844239e81903d07b732ab","value":147}},"f7de2c8b9f1a4baf87a31cbadeb6424f":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ba343c49657048ad9cdd035090b9b590","placeholder":"​","style":"IPY_MODEL_691c2eaf31b84e9c89458b22160e4df5","value":" 147/147 [00:00&lt;00:00, 5.12kB/s]"}},"a8ea02156db1495999105c450d4172bf":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d6136a686f1f4b7da80d48e12d0bc79c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a8e477092733491f9ece5528a272652d":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a4f1cf289630449999dc9c279ad069aa":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2aa24b5ff19844239e81903d07b732ab":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"ba343c49657048ad9cdd035090b9b590":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"691c2eaf31b84e9c89458b22160e4df5":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","source":["# Instalacja pakietów"],"metadata":{"id":"ZXsOR6oJOJbd"}},{"cell_type":"code","execution_count":2,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"8l0hzptKNiZS","executionInfo":{"status":"ok","timestamp":1676652341429,"user_tz":-60,"elapsed":5296,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"outputId":"c8f3b15a-78e6-4b24-d7f8-7a76ef36cc76"},"outputs":[{"output_type":"stream","name":"stdout","text":["Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n","Requirement already satisfied: transformers in /usr/local/lib/python3.8/dist-packages (4.26.1)\n","Requirement already satisfied: datasets in /usr/local/lib/python3.8/dist-packages (2.9.0)\n","Requirement already satisfied: torch in /usr/local/lib/python3.8/dist-packages (1.13.1+cu116)\n","Requirement already satisfied: sentencepiece in /usr/local/lib/python3.8/dist-packages (0.1.97)\n","Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (1.21.6)\n","Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (0.12.1)\n","Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.8/dist-packages (from transformers) (4.64.1)\n","Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from transformers) (2.25.1)\n","Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (6.0)\n","Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (2022.6.2)\n","Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (0.13.2)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers) (3.9.0)\n","Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (23.0)\n","Requirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.8/dist-packages (from datasets) (2023.1.0)\n","Requirement already satisfied: pyarrow>=6.0.0 in /usr/local/lib/python3.8/dist-packages (from datasets) (9.0.0)\n","Requirement already satisfied: pandas in /usr/local/lib/python3.8/dist-packages (from datasets) (1.3.5)\n","Requirement already satisfied: responses<0.19 in /usr/local/lib/python3.8/dist-packages (from datasets) (0.18.0)\n","Requirement already satisfied: xxhash in /usr/local/lib/python3.8/dist-packages (from datasets) (3.2.0)\n","Requirement already satisfied: dill<0.3.7 in /usr/local/lib/python3.8/dist-packages (from datasets) (0.3.6)\n","Requirement already satisfied: multiprocess in /usr/local/lib/python3.8/dist-packages (from datasets) (0.70.14)\n","Requirement already satisfied: aiohttp in /usr/local/lib/python3.8/dist-packages (from datasets) (3.8.3)\n","Requirement already satisfied: typing-extensions in /usr/local/lib/python3.8/dist-packages (from torch) (4.4.0)\n","Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.8.2)\n","Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (2.1.1)\n","Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (4.0.2)\n","Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (22.2.0)\n","Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.3)\n","Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (6.0.4)\n","Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.1)\n","Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2.10)\n","Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (1.26.14)\n","Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (4.0.0)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2022.12.7)\n","Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas->datasets) (2.8.2)\n","Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas->datasets) (2022.7.1)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil>=2.7.3->pandas->datasets) (1.15.0)\n"]}],"source":["!pip install transformers datasets torch sentencepiece"]},{"cell_type":"markdown","source":["# Załadowanie datasetu"],"metadata":{"id":"dhN0rmb5Oi3d"}},{"cell_type":"code","source":["from datasets import load_dataset"],"metadata":{"id":"tnaDkwZ2Pbnn","executionInfo":{"status":"ok","timestamp":1676652341811,"user_tz":-60,"elapsed":389,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}}},"execution_count":3,"outputs":[]},{"cell_type":"code","source":["dataset = load_dataset(\"sms_spam\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":211,"referenced_widgets":["ba9d40cd07c34f4fb1f9fbd73092fbf6","9de313fab7ae49b080fb5192acb456c0","211c322124ff451c8d6cddbc842e650b","9e285551432c4b5daa6afa6049a0a88d","e355e0d5122a46d785cdc07117a66990","2db46abba29f4f9789de6fdbec1af7a0","9bff66fdf5c94944a63c5525685a71af","27f2295f6b0c4e2f97f511514901b60d","24dd9a57eb984bdaa61c77601a0f5052","752ab7a625c44966b4d0204355ed1d03","e575215f324e4f959cee7086a32c4039","7ae5fccd7dc442f0b6edf652210e8f45","a976451070bc43d3a5f2504e6b2441b6","842cb52b1c214db9a5704752497d5944","20dd392ebbbf4027827a722f1b5244de","b2f23e1760284b5f8b6428d499556825","dc6f0fbc82a84b6f9e01e085a66260b9","36540af07eb34debbc483e635a1aa978","67a513749d4d4e6f8fce80debaafe1a9","ab6920be0d57483db97d91d30b92bc0a","b3cd7dbebf6b488a8a4d7146e0dc89ec","620ed6142c3e4a96b49ce51dca45dbb4","c4ba45f6ee0e4edb9cb1ee26fadeb01a","f87e37e286a74e94adc773e74c2c09c9","c6ab86832ff94f818329bd34d6e6efe2","4c8fbd675bd24a279348316ad06b6676","abb70e15da2849edbc926078009f19de","e147057451f844de84f8c9eb7896550f","d9fd300cff5a4b5d97cddcb217677465","5e95b5a9e0724361a84b729b547ca3ad","148d65aedcb14ae59b2b79363ed919f7","37e3971106aa456981303debb9b4c225","0d66ab66ba6744f99ec4c63334eeb106","0ef3d6b2ea5542eea95c2a8058652cde","834d071dc2844826afe2cba68d7b7ee0","ddc9057d5fcb4f06ad92e89181850c2b","3d6f5d5597f3442ca8d50e82c37ea8f0","7c0f4fac68fc49f8b288d8348196ce73","7317d2612bf54f849071aa6c2846d2a0","a85008da935b414dba2c9cec5a81d998","7c15d362b51c4466a9d994208a1058b7","1e53fe4a87fc4ce3bdf2145fb1f1db7a","328ff44536394df59f5aad906b4ba472","0abafc781cd14f36af17144f84b4e0ee","b0e9810c15bf4877b12cbe5b2b391c11","b11ff7e8103d4f19808023ac61f1c495","a1e2e0b7b94b4c29b10e256d4665ac1c","f6308b3e42ee4759982e054f7f1fc13b","a525b423a38446e7acd4adf0806ce16b","d777deca350c43659ae1c9de6809f975","5b874a9c118b4aba94fb8df333829703","d907fcfb69d44b9585c46df7b97980cc","6b0be17848444747b51137f20854ad53","51f48056fa734f969b98d6779c75fda4","1a7a53fd227441e7bdcd31050b189a85","79b64a59aa904f46803c0013bfaa67c7","8c09a31ddcd94408baea9bb143b3608f","2af0dd8e534d4e25a89ac415d3f7a536","114db7aaead5484ba5379731f0f74de5","1f293d6ea070431e90f8ad1a79c42822","aed3d46ba465417cab5d75d09d395d54","5246a27a3ea94e9cb6d1baafe4a51cdd","2d5a8b6f1e70487781a88da6dc8b6aa5","90b9696a21794d99b9d4dcd34ec7866e","f1221b8b45f14027855dc31cbc932adc","1836db5096354e18810611ffd14c39b8"]},"id":"cCiAuRqrOkvV","executionInfo":{"status":"ok","timestamp":1676652344210,"user_tz":-60,"elapsed":2403,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"outputId":"e3b61204-b6b5-4d4e-f66f-48cde363888b"},"execution_count":4,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading builder script:   0%|          | 0.00/3.21k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"ba9d40cd07c34f4fb1f9fbd73092fbf6"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading metadata:   0%|          | 0.00/1.69k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"7ae5fccd7dc442f0b6edf652210e8f45"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading readme:   0%|          | 0.00/4.87k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"c4ba45f6ee0e4edb9cb1ee26fadeb01a"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Downloading and preparing dataset sms_spam/plain_text to /root/.cache/huggingface/datasets/sms_spam/plain_text/1.0.0/53f051d3b5f62d99d61792c91acefe4f1577ad3e4c216fb0ad39e30b9f20019c...\n"]},{"output_type":"display_data","data":{"text/plain":["Downloading data:   0%|          | 0.00/203k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"0ef3d6b2ea5542eea95c2a8058652cde"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Generating train split:   0%|          | 0/5574 [00:00<?, ? examples/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"b0e9810c15bf4877b12cbe5b2b391c11"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Dataset sms_spam downloaded and prepared to /root/.cache/huggingface/datasets/sms_spam/plain_text/1.0.0/53f051d3b5f62d99d61792c91acefe4f1577ad3e4c216fb0ad39e30b9f20019c. Subsequent calls will reuse this data.\n"]},{"output_type":"display_data","data":{"text/plain":["  0%|          | 0/1 [00:00<?, ?it/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"79b64a59aa904f46803c0013bfaa67c7"}},"metadata":{}}]},{"cell_type":"code","source":["dataset['train'][0]"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"JKFHPko3OnAV","executionInfo":{"status":"ok","timestamp":1676652344211,"user_tz":-60,"elapsed":14,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"outputId":"996c303d-4a72-4c59-8b4a-02bdd32396c9"},"execution_count":5,"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\\n',\n"," 'label': 0}"]},"metadata":{},"execution_count":5}]},{"cell_type":"markdown","source":["# Przygotowanie datasetu"],"metadata":{"id":"l140vJrgYxPr"}},{"cell_type":"code","source":["parsed_dataset = []\n","\n","for row in dataset['train']:\n","  text = row['sms'].replace(\"\\n\", \"\")\n","  new_row = {}\n","  new_row['sms'] = text\n","  if row['label'] == 0:\n","    new_row['label'] = \"False\"\n","  else:\n","    new_row['label'] = \"True\"\n","  parsed_dataset.append(new_row)\n","\n","parsed_dataset[0]"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"1boUF-YiY3_y","executionInfo":{"status":"ok","timestamp":1676652344211,"user_tz":-60,"elapsed":11,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"outputId":"ae00c79a-a17e-46e3-8cbd-2165042170d2"},"execution_count":6,"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...',\n"," 'label': 'False'}"]},"metadata":{},"execution_count":6}]},{"cell_type":"markdown","source":["# Tokenizer FLAN-T5"],"metadata":{"id":"O-J-jBDxPJcn"}},{"cell_type":"code","source":["from transformers import AutoTokenizer"],"metadata":{"id":"P23AYPX1PZ6g","executionInfo":{"status":"ok","timestamp":1676652347327,"user_tz":-60,"elapsed":3122,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}}},"execution_count":7,"outputs":[]},{"cell_type":"code","source":["tokenizer = AutoTokenizer.from_pretrained('google/flan-t5-base')"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":145,"referenced_widgets":["130f4a8f140d43e9993f6f158d643f91","052168c2faa04049bec6605ec9fc654a","8baade75913f4bc28033f916db729edd","9032dbaf99f04ef8a410969b7345928f","d703073401d84414b90c3d71f9914e39","6178481579ef4b2086d730108c77132e","81761b679c9f462a907ee227ca4e0e3d","cad9c06510c44d76b8873ab1f1b9e0ce","da24d4f940734731980bbf4511b7f388","0137abcc3a8c4a7fbc77dfc578f8bd28","47340075461a44d981ecea3883273a3c","09f475e7464c4101a94dce60d034ac12","5244aabb420f4970954781124f07ee1c","c477c1291af84638a961dcf1d43a0eb5","9dd0b870a8da4380af8e15c6e4f02dae","d1c3f34e830841c39e7c86f754843903","e42f6346d52346779e2ec76061abf4db","6a53778d0eb041959e476ead69b761ad","4831844601e548849e3c3ae6c5c0f734","9ad9b94a8d5b41a99fae31fac4782dae","1a8502e4909b493e8a0015c9406250a2","faed3ee84d1649f6aee33545623c3414","bf8558287dc54fe19fd4eca5043dc182","bb3022078d654d90aad064587d5fcb81","79d8d0f473d54e518023d13b4896dcb0","1cdf0415e5234288a5ed6900a61bceb1","ef9cd89580534dea85fcd3ef8e129d37","0bd704ed93f249a882ea09a9b4760856","bb343e8421494468ba70a338e46ce97b","b5a1ca49058e4988a6a8f4f072bd8bb6","360ee70e58784c4ab7a08333afd4ff30","e51e0982800a4dc3bea1feeab7e3b5f1","72f4d1baccca4021b6392763c3f6068d","453344c1de7c4fef9ad94d31fd1c4972","afa1d46f350f439ea7371b29b11f12ee","29ce5457214f4b7cbac8a866ec805fa0","c9f71fbae55448d69d13216a4a88f040","fc0695782e4f4ddc8c3b9140bef9a482","0bf540f48dba48eaa2ef411032e8372c","bf7f88a03d1e4bb68ca7e3df611525ba","8d82e208f6b44e409050891ab76f1551","2ba133a583514a528fae49101dddd523","6e1f83ef1afc42e0a0d8562c1f871067","c0194dbf3d5842fa8c1437def7113be6"]},"id":"q5Jz0E_oPMBr","executionInfo":{"status":"ok","timestamp":1676652348258,"user_tz":-60,"elapsed":939,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"outputId":"a828b2ce-c419-49b0-dc1d-d6a3721021f4"},"execution_count":8,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading (…)okenizer_config.json:   0%|          | 0.00/2.54k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"130f4a8f140d43e9993f6f158d643f91"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)\"spiece.model\";:   0%|          | 0.00/792k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"09f475e7464c4101a94dce60d034ac12"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)/main/tokenizer.json:   0%|          | 0.00/2.42M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"bf8558287dc54fe19fd4eca5043dc182"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)cial_tokens_map.json:   0%|          | 0.00/2.20k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"453344c1de7c4fef9ad94d31fd1c4972"}},"metadata":{}}]},{"cell_type":"code","source":["sms = parsed_dataset[0]['sms']\n","print('Original: ', sms)\n","print('Tokenized: ', tokenizer.tokenize(sms))\n","print('Token IDs: ', tokenizer.convert_tokens_to_ids(tokenizer.tokenize(sms)))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"dfxJQpoePsvI","executionInfo":{"status":"ok","timestamp":1676652348732,"user_tz":-60,"elapsed":477,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"outputId":"87fcb627-290c-4ebf-8712-53ff89d251ba"},"execution_count":9,"outputs":[{"output_type":"stream","name":"stdout","text":["Original:  Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\n","Tokenized:  ['▁Go', '▁until', '▁jur', 'ong', '▁point', ',', '▁crazy', '.', '.', '▁Available', '▁only', '▁in', '▁bug', 'is', '▁', 'n', '▁great', '▁world', '▁la', '▁', 'e', '▁buffet', '...', '▁Cine', '▁there', '▁got', '▁', 'a', 'more', '▁wa', 't', '...']\n","Token IDs:  [1263, 552, 10081, 2444, 500, 6, 6139, 5, 5, 8144, 163, 16, 8143, 159, 3, 29, 248, 296, 50, 3, 15, 15385, 233, 17270, 132, 530, 3, 9, 3706, 8036, 17, 233]\n"]}]},{"cell_type":"markdown","source":["# Few shot learning"],"metadata":{"id":"UpluhM8cU5Ir"}},{"cell_type":"code","source":["print(parsed_dataset[0]) #0\n","print(parsed_dataset[123]) #1\n","print(parsed_dataset[2000]) #0\n","print(parsed_dataset[3002]) #1"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"7uNUkixPU85O","executionInfo":{"status":"ok","timestamp":1676652348733,"user_tz":-60,"elapsed":23,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"outputId":"b9db82da-7c31-4d59-b46f-8a49789e2d44"},"execution_count":10,"outputs":[{"output_type":"stream","name":"stdout","text":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...', 'label': 'False'}\n","{'sms': 'Todays Voda numbers ending 7548 are selected to receive a $350 award. If you have a match please call 08712300220 quoting claim code 4041 standard rates app', 'label': 'True'}\n","{'sms': \"LMAO where's your fish memory when I need it?\", 'label': 'False'}\n","{'sms': 'This message is free. Welcome to the new & improved Sex & Dogging club! To unsubscribe from this service reply STOP. msgs@150p 18+only', 'label': 'True'}\n"]}]},{"cell_type":"code","source":["non_spam_1 = \"Content of the text message: \" + parsed_dataset[0]['sms'] + \"\\nAnswer if this text message is spam or not: False\\n\\n\"\n","spam_1 = \"Content of the text message: \" + parsed_dataset[123]['sms'] + \"\\nAnswer if this text message is spam or not: True\\n\\n\"\n","non_spam_2 = \"Content of the text message: \" + parsed_dataset[2000]['sms'] + \"\\nAnswer if this text message is spam or not: False\\n\\n\"\n","spam_2 = \"Content of the text message: \" + parsed_dataset[3002]['sms'] + \"\\nAnswer if this text message is spam or not: True\\n\\n\"\n","\n","few_shot_prefix = non_spam_1 + spam_1 + non_spam_2 + spam_2 + \"Content of the text message: \"\n","print(few_shot_prefix)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"lj0issBznZfK","executionInfo":{"status":"ok","timestamp":1676652348733,"user_tz":-60,"elapsed":16,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"outputId":"2012323d-eb35-4494-e8c6-6051963e9899"},"execution_count":11,"outputs":[{"output_type":"stream","name":"stdout","text":["Content of the text message: Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\n","Answer if this text message is spam or not: False\n","\n","Content of the text message: Todays Voda numbers ending 7548 are selected to receive a $350 award. If you have a match please call 08712300220 quoting claim code 4041 standard rates app\n","Answer if this text message is spam or not: True\n","\n","Content of the text message: LMAO where's your fish memory when I need it?\n","Answer if this text message is spam or not: False\n","\n","Content of the text message: This message is free. Welcome to the new & improved Sex & Dogging club! To unsubscribe from this service reply STOP. msgs@150p 18+only\n","Answer if this text message is spam or not: True\n","\n","Content of the text message: \n"]}]},{"cell_type":"markdown","source":["# Load FLAN-T5 model"],"metadata":{"id":"okTx_ynMV0rH"}},{"cell_type":"code","source":["from transformers import AutoModelForSeq2SeqLM"],"metadata":{"id":"Eu-7Eed8WgN0","executionInfo":{"status":"ok","timestamp":1676652348734,"user_tz":-60,"elapsed":11,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}}},"execution_count":12,"outputs":[]},{"cell_type":"code","source":["model = AutoModelForSeq2SeqLM.from_pretrained('google/flan-t5-base')\n","\n","model.cuda()"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000,"referenced_widgets":["8b2028bc0dd34cd49ff063356924f2d1","b61f9f83affd4bffa20e35550e1dcf11","9402536c51f54eebb6f9d6ebff692ed3","27d09ed90c3048ef92deab33ed7b1b5c","fda0aada96374e159ddfad0831e795d5","81ea8a9d70a8416db4d60b4596e00f92","c2f2cf8567514612a788b4ade694daba","a771d23698b641e09bbfc51d004a7358","28ae83bc70ba41d7aad0f785d79e6600","032276c9596f4100965fb757c5741709","04be2018587746949e3def7c3a51d7a9","62e2e7c6fc834eca82be7f85c9fa9701","060b98b462b84394b4aa833f31abcc69","e572c1b7e70b4abc97e390ad78ac6f3f","3eb033bb16ce4534a7f7538c740c1d39","a38561dd1f81429d9292affd976070e0","2097b347701f4eda889728f50f62252f","3276c774bee745a791d14b3d7490cc8c","a47dba0fd1f04505927b837d252fe5fe","b4e93db0bd734f9ebccc281be866fede","6a8020dba5c94c69a4c39abc890b877e","20edf60841f54c5b9b5863a20a9ea0a5","6eb421544bdd4100b92e2dc12e783485","c3ee8e64d4024205896cb81a77d08733","29d772f88bc3404585377162d5762272","f7de2c8b9f1a4baf87a31cbadeb6424f","a8ea02156db1495999105c450d4172bf","d6136a686f1f4b7da80d48e12d0bc79c","a8e477092733491f9ece5528a272652d","a4f1cf289630449999dc9c279ad069aa","2aa24b5ff19844239e81903d07b732ab","ba343c49657048ad9cdd035090b9b590","691c2eaf31b84e9c89458b22160e4df5"]},"id":"JKv9O8kfV2zZ","executionInfo":{"status":"ok","timestamp":1676652375456,"user_tz":-60,"elapsed":26732,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"outputId":"7e410ecf-24f4-40fe-dd54-8c4ca1012f05"},"execution_count":13,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading (…)lve/main/config.json:   0%|          | 0.00/1.40k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"8b2028bc0dd34cd49ff063356924f2d1"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)\"pytorch_model.bin\";:   0%|          | 0.00/990M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"62e2e7c6fc834eca82be7f85c9fa9701"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)neration_config.json:   0%|          | 0.00/147 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6eb421544bdd4100b92e2dc12e783485"}},"metadata":{}},{"output_type":"execute_result","data":{"text/plain":["T5ForConditionalGeneration(\n","  (shared): Embedding(32128, 768)\n","  (encoder): T5Stack(\n","    (embed_tokens): Embedding(32128, 768)\n","    (block): ModuleList(\n","      (0): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","              (relative_attention_bias): Embedding(32, 12)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (1): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (2): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (3): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (4): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (5): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (6): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (7): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (8): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (9): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (10): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (11): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","    )\n","    (final_layer_norm): T5LayerNorm()\n","    (dropout): Dropout(p=0.1, inplace=False)\n","  )\n","  (decoder): T5Stack(\n","    (embed_tokens): Embedding(32128, 768)\n","    (block): ModuleList(\n","      (0): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","              (relative_attention_bias): Embedding(32, 12)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (1): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (2): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (3): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (4): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (5): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (6): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (7): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (8): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (9): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (10): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (11): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","    )\n","    (final_layer_norm): T5LayerNorm()\n","    (dropout): Dropout(p=0.1, inplace=False)\n","  )\n","  (lm_head): Linear(in_features=768, out_features=32128, bias=False)\n",")"]},"metadata":{},"execution_count":13}]},{"cell_type":"markdown","source":["# Helper functions"],"metadata":{"id":"F_SDAwxoawDy"}},{"cell_type":"code","source":["import torch"],"metadata":{"id":"rdWMg_KJZEZH","executionInfo":{"status":"ok","timestamp":1676652375456,"user_tz":-60,"elapsed":8,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}}},"execution_count":14,"outputs":[]},{"cell_type":"code","source":["def calculate_accuracy(preds, target):\n","  results_ok = 0.0\n","  results_false = 0.0\n","\n","  for idx, pred in enumerate(preds):\n","    if pred == target[idx]:\n","      results_ok += 1.0\n","    else:\n","      results_false += 1.0\n","\n","  return results_ok / (results_ok + results_false)"],"metadata":{"id":"FzUi8908ax61","executionInfo":{"status":"ok","timestamp":1676652375457,"user_tz":-60,"elapsed":7,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}}},"execution_count":15,"outputs":[]},{"cell_type":"code","source":["if torch.cuda.is_available():     \n","    device = torch.device(\"cuda\")\n","\n","    print('There are %d GPU(s) available.' % torch.cuda.device_count())\n","    print('We will use the GPU:', torch.cuda.get_device_name(0))\n","\n","else:\n","    print('No GPU available, using the CPU instead.')\n","    device = torch.device(\"cpu\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"86i7iRmtW-6L","executionInfo":{"status":"ok","timestamp":1676652375457,"user_tz":-60,"elapsed":6,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"outputId":"f2f5e476-8fed-4d32-e80a-72f33b07f8af"},"execution_count":16,"outputs":[{"output_type":"stream","name":"stdout","text":["There are 1 GPU(s) available.\n","We will use the GPU: Tesla T4\n"]}]},{"cell_type":"markdown","source":["# Predykcja"],"metadata":{"id":"H_YI3bS3VHQE"}},{"cell_type":"code","source":["parsed_dataset = parsed_dataset[1:123] + parsed_dataset[124:2000] + parsed_dataset[2001:3002] + parsed_dataset[3003:]\n","predictions = []\n","expected = []\n","\n","for row in parsed_dataset:\n","  input_text = few_shot_prefix + row['sms'] + \"\\nAnswer if this text message is spam or not: \"\n","  input_ids = tokenizer(input_text, return_tensors=\"pt\").input_ids.to(device)\n","\n","  generated_ids = model.generate(input_ids, do_sample=True, temperature=0.9, max_length=400)\n","  generated_text = tokenizer.decode(generated_ids[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)\n","  \n","  predictions.append(generated_text)\n","  expected.append(row['label'])\n","\n","acc = calculate_accuracy(predictions, expected)\n","print(acc)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"vunLGuBGVGmh","executionInfo":{"status":"ok","timestamp":1676652961996,"user_tz":-60,"elapsed":432382,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"outputId":"7421dcc1-592b-484c-f8ad-b3ea88805c3d"},"execution_count":18,"outputs":[{"output_type":"stream","name":"stdout","text":["0.503593244699964\n"]}]},{"cell_type":"code","source":["print(\"Sample prediction: {}, expected: {}\".format(predictions[101], expected[101]))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"20WbYZmLaDl7","executionInfo":{"status":"ok","timestamp":1676652961997,"user_tz":-60,"elapsed":18,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"outputId":"9a0f5f23-78ef-48b3-f6cc-4c64aad653ed"},"execution_count":19,"outputs":[{"output_type":"stream","name":"stdout","text":["Sample prediction: False, expected: False\n"]}]},{"cell_type":"markdown","source":["# MCC Score"],"metadata":{"id":"dLYc9WXz_B1o"}},{"cell_type":"code","source":["from sklearn.metrics import matthews_corrcoef\n","\n","print('Calculating Matthews Corr. Coef. for each batch...')\n","matthews = matthews_corrcoef(expected, predictions)    \n","print('Total MCC: %.3f' % matthews)"],"metadata":{"id":"hPEPpXXX_DXR","executionInfo":{"status":"ok","timestamp":1676652962709,"user_tz":-60,"elapsed":722,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"e5035c3e-32eb-438b-aa77-3c7d6db51b53"},"execution_count":20,"outputs":[{"output_type":"stream","name":"stdout","text":["Calculating Matthews Corr. Coef. for each batch...\n","Total MCC: 0.010\n"]}]},{"cell_type":"markdown","source":["# Save model"],"metadata":{"id":"GPhCp068_Iwq"}},{"cell_type":"code","source":["from google.colab import drive\n","\n","drive.mount('/content/gdrive/', force_remount=True)\n","\n","output_dir = '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_1_FLAN-T5_Model'\n","print(\"Saving model to %s\" % output_dir)\n","\n","model_to_save = model.module if hasattr(model, 'module') else model\n","model_to_save.save_pretrained(output_dir)\n","tokenizer.save_pretrained(output_dir)"],"metadata":{"id":"avafCMoS_KDF","executionInfo":{"status":"ok","timestamp":1676653169797,"user_tz":-60,"elapsed":207094,"user":{"displayName":"Mikołaj Paterka","userId":"09090979117369578839"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"f8143c19-99cd-4e13-d40c-918f5b2c8941"},"execution_count":21,"outputs":[{"output_type":"stream","name":"stdout","text":["Mounted at /content/gdrive/\n","Saving model to /content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_1_FLAN-T5_Model\n"]},{"output_type":"execute_result","data":{"text/plain":["('/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_1_FLAN-T5_Model/tokenizer_config.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_1_FLAN-T5_Model/special_tokens_map.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_1_FLAN-T5_Model/spiece.model',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_1_FLAN-T5_Model/added_tokens.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_1_FLAN-T5_Model/tokenizer.json')"]},"metadata":{},"execution_count":21}]},{"cell_type":"markdown","source":["# Bibliografia\n","- https://huggingface.co/docs/transformers/main/en/model_doc/flan-t5\n","- https://mccormickml.com/2019/07/22/BERT-fine-tuning/#a1-saving--loading-fine-tuned-model\n","- https://huggingface.co/docs/transformers/model_doc/t5#training"],"metadata":{"id":"wHzm2_nDA6i-"}}]}
\ No newline at end of file
diff --git a/projekt/Ver3_FLAN-T5_sms_spam.ipynb b/projekt/Ver3_FLAN-T5_sms_spam.ipynb
new file mode 100644
index 0000000..9a237fb
--- /dev/null
+++ b/projekt/Ver3_FLAN-T5_sms_spam.ipynb
@@ -0,0 +1 @@
+{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"},"gpuClass":"standard","widgets":{"application/vnd.jupyter.widget-state+json":{"25db59aa53d542bfb6d54da340006fcd":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_2eaefa22c2e14c40a2de9592676a8d82","IPY_MODEL_1bf81e50052942208c5100bfdf0c985a","IPY_MODEL_5422e114814145d1a85b423b93d4133c"],"layout":"IPY_MODEL_0031c9a8c8724bce9aff6b3cafc94c2b"}},"2eaefa22c2e14c40a2de9592676a8d82":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3d3b84701cbf468c96170f7563e42408","placeholder":"​","style":"IPY_MODEL_96bdc942fcca434798c89e4ec74b6dce","value":"Downloading builder script: 100%"}},"1bf81e50052942208c5100bfdf0c985a":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_5cabe61f114a4c33ae677d9b5306c635","max":3208,"min":0,"orientation":"horizontal","style":"IPY_MODEL_6b39afc3b8bc43768b376186aafe96de","value":3208}},"5422e114814145d1a85b423b93d4133c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7bd83463f6504145b0ca14ab78750d3a","placeholder":"​","style":"IPY_MODEL_61d94df2723e4e2d94cd74a1b884d409","value":" 3.21k/3.21k [00:00&lt;00:00, 81.9kB/s]"}},"0031c9a8c8724bce9aff6b3cafc94c2b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3d3b84701cbf468c96170f7563e42408":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"96bdc942fcca434798c89e4ec74b6dce":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5cabe61f114a4c33ae677d9b5306c635":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6b39afc3b8bc43768b376186aafe96de":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"7bd83463f6504145b0ca14ab78750d3a":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"61d94df2723e4e2d94cd74a1b884d409":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d160c37c8eb241eda89f7713fd4d05c2":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_5fa8d14ffdbc45e6812b50e0d27e14fe","IPY_MODEL_a54fa4aedddc46ba84224971eb1e5227","IPY_MODEL_98c7b45870cc49b7bb3addd225e04301"],"layout":"IPY_MODEL_6f3ada40efe54c13b2c14ad60a6b18e4"}},"5fa8d14ffdbc45e6812b50e0d27e14fe":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f82774bb70cc4f90b80d723f8aa9c7e4","placeholder":"​","style":"IPY_MODEL_b3f23480d62d4c50a8aaa03118f6b5ce","value":"Downloading metadata: 100%"}},"a54fa4aedddc46ba84224971eb1e5227":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_3a9cfb628ef04d8ca222584f40be05c4","max":1687,"min":0,"orientation":"horizontal","style":"IPY_MODEL_9bdb6d0c6fa54454957211029a960159","value":1687}},"98c7b45870cc49b7bb3addd225e04301":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_67ab4e3216154b67bcdac51d4548ed75","placeholder":"​","style":"IPY_MODEL_524913d4b3ca4f729acfc975ea490b46","value":" 1.69k/1.69k [00:00&lt;00:00, 81.2kB/s]"}},"6f3ada40efe54c13b2c14ad60a6b18e4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f82774bb70cc4f90b80d723f8aa9c7e4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b3f23480d62d4c50a8aaa03118f6b5ce":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3a9cfb628ef04d8ca222584f40be05c4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9bdb6d0c6fa54454957211029a960159":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"67ab4e3216154b67bcdac51d4548ed75":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"524913d4b3ca4f729acfc975ea490b46":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a543b6e11ab046748cdd6c946a1f9084":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8c2521b501c64c90998bc05d1a76281c","IPY_MODEL_bca85efcfcd546b9a43ce79a68b7f019","IPY_MODEL_0f359da3c8b1402b8e391c2360531d19"],"layout":"IPY_MODEL_1a6b59d6f8e54a30b69822e6b59fd99f"}},"8c2521b501c64c90998bc05d1a76281c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_dccb6a9167aa4b60898047d46363d80e","placeholder":"​","style":"IPY_MODEL_545f9b3e11cf42818c25709159e7fb3e","value":"Downloading readme: 100%"}},"bca85efcfcd546b9a43ce79a68b7f019":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_6af88f49ff5346b9a0ad07faceede0bd","max":4872,"min":0,"orientation":"horizontal","style":"IPY_MODEL_fc87784596044dd78d045989dbb53a91","value":4872}},"0f359da3c8b1402b8e391c2360531d19":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_331e1896aa424786a810a711143d478e","placeholder":"​","style":"IPY_MODEL_a39965de69714b26a78f8b976a33a42e","value":" 4.87k/4.87k [00:00&lt;00:00, 312kB/s]"}},"1a6b59d6f8e54a30b69822e6b59fd99f":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"dccb6a9167aa4b60898047d46363d80e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"545f9b3e11cf42818c25709159e7fb3e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6af88f49ff5346b9a0ad07faceede0bd":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fc87784596044dd78d045989dbb53a91":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"331e1896aa424786a810a711143d478e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a39965de69714b26a78f8b976a33a42e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"80a4187b45134a4ab30a7748e4f6ff28":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_326f7ae8ea614699a30da8dfe3f929be","IPY_MODEL_c7574a234b8d492bba0fec988d8e8ae1","IPY_MODEL_2282de3148204153873e6c678d3ab97d"],"layout":"IPY_MODEL_9a9392cc5903455e9206d9c965f5005c"}},"326f7ae8ea614699a30da8dfe3f929be":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0c48c929e21f4858b9323b7f4ac220b2","placeholder":"​","style":"IPY_MODEL_17dd8779c5d642ceaa85e0d7c8524436","value":"Downloading data: 100%"}},"c7574a234b8d492bba0fec988d8e8ae1":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_e5e5197e03c040a99a5d31c57f8a047c","max":203415,"min":0,"orientation":"horizontal","style":"IPY_MODEL_a9aa4d15ad5143b1a5686972ef47186e","value":203415}},"2282de3148204153873e6c678d3ab97d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d23808437da04c37868bd7526e685a5c","placeholder":"​","style":"IPY_MODEL_6ac94e8230564af9b05ac561ab5d8e7b","value":" 203k/203k [00:00&lt;00:00, 1.02MB/s]"}},"9a9392cc5903455e9206d9c965f5005c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0c48c929e21f4858b9323b7f4ac220b2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"17dd8779c5d642ceaa85e0d7c8524436":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"e5e5197e03c040a99a5d31c57f8a047c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a9aa4d15ad5143b1a5686972ef47186e":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"d23808437da04c37868bd7526e685a5c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6ac94e8230564af9b05ac561ab5d8e7b":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4f03235725824725add5a46e03a5156e":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_af2ac44def004870a388284503a45b07","IPY_MODEL_3a70f4da67754c5796c8ab5e8eaec187","IPY_MODEL_1c0621a41647430d8dfda34f2f7d8e2f"],"layout":"IPY_MODEL_d0d7c235a53e46e98f99229f26b665d0"}},"af2ac44def004870a388284503a45b07":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_accb4a02861c4ad3a7965ba707ab8a38","placeholder":"​","style":"IPY_MODEL_7d7810f28a2342848026eb36e967d8a5","value":"Generating train split:  62%"}},"3a70f4da67754c5796c8ab5e8eaec187":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"","description_tooltip":null,"layout":"IPY_MODEL_f7f34fc698bc416383ae59828301469d","max":5574,"min":0,"orientation":"horizontal","style":"IPY_MODEL_373bf37aada844a187f13af6d723333b","value":5574}},"1c0621a41647430d8dfda34f2f7d8e2f":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7f309cfb6bc54a5e8872524eb63354c2","placeholder":"​","style":"IPY_MODEL_c69df495e9324100b87aae91c9b34bf6","value":" 3431/5574 [00:00&lt;00:00, 24883.55 examples/s]"}},"d0d7c235a53e46e98f99229f26b665d0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":"hidden","width":null}},"accb4a02861c4ad3a7965ba707ab8a38":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7d7810f28a2342848026eb36e967d8a5":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f7f34fc698bc416383ae59828301469d":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"373bf37aada844a187f13af6d723333b":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"7f309cfb6bc54a5e8872524eb63354c2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c69df495e9324100b87aae91c9b34bf6":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"209b3c2d088b4512bf513e17e74e8a30":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_eef55327290e4e98b5ac4b2c3e8b4ad4","IPY_MODEL_495b6416c57149bfa9d8418676c4541d","IPY_MODEL_4c01dbed470a4fbc94aaedc3c679385f"],"layout":"IPY_MODEL_dda3f23953f64ceead909262eb0f629d"}},"eef55327290e4e98b5ac4b2c3e8b4ad4":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6165a4a6a5454cd8b73c9702771dcae1","placeholder":"​","style":"IPY_MODEL_ff4d3372dcc14403aeb19ff960a518f5","value":"100%"}},"495b6416c57149bfa9d8418676c4541d":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_dea72bac03c1492da956598f66465472","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_4b87d5b0da3c4a6484d1757a09de382b","value":1}},"4c01dbed470a4fbc94aaedc3c679385f":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_fe6080acb5544dd8a3c512fbf38afd87","placeholder":"​","style":"IPY_MODEL_1b4bf9de6c59492091f84d80871545c5","value":" 1/1 [00:00&lt;00:00, 18.02it/s]"}},"dda3f23953f64ceead909262eb0f629d":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6165a4a6a5454cd8b73c9702771dcae1":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ff4d3372dcc14403aeb19ff960a518f5":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"dea72bac03c1492da956598f66465472":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4b87d5b0da3c4a6484d1757a09de382b":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"fe6080acb5544dd8a3c512fbf38afd87":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1b4bf9de6c59492091f84d80871545c5":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d5c9c4fcc72f46639246e0e733af5ff8":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8b8719c742754a15baa3c14a9f2f1852","IPY_MODEL_b309af1852fc4d8d8985f2690b768485","IPY_MODEL_30237ebd73554b428d13a66f18d90069"],"layout":"IPY_MODEL_e80a747c0d934144aa2b78cff101c87a"}},"8b8719c742754a15baa3c14a9f2f1852":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_494c694bd92e49d3831d716cb90b4dd8","placeholder":"​","style":"IPY_MODEL_e03d7f43bdbc4deda2c35eaaece29fe2","value":"Downloading (…)okenizer_config.json: 100%"}},"b309af1852fc4d8d8985f2690b768485":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_56aa37b752a54936a09e2167f0d2c281","max":2537,"min":0,"orientation":"horizontal","style":"IPY_MODEL_492a083bfe674cd496a91fc8c51a4646","value":2537}},"30237ebd73554b428d13a66f18d90069":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_58865aaf52a94747b20aac75ebb46426","placeholder":"​","style":"IPY_MODEL_6a609ebaee3b465fa35107769eb4cc9d","value":" 2.54k/2.54k [00:00&lt;00:00, 43.2kB/s]"}},"e80a747c0d934144aa2b78cff101c87a":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"494c694bd92e49d3831d716cb90b4dd8":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e03d7f43bdbc4deda2c35eaaece29fe2":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"56aa37b752a54936a09e2167f0d2c281":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"492a083bfe674cd496a91fc8c51a4646":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"58865aaf52a94747b20aac75ebb46426":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6a609ebaee3b465fa35107769eb4cc9d":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6a23888d5c8a4080b02e6b632f14c6ef":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_a3bc16f6a15e4fd4b377c59efb7c044c","IPY_MODEL_5efe04cb883e45f0bbdfbbd3e395b15c","IPY_MODEL_0dc054f35b374f878a24d535bdfad9fb"],"layout":"IPY_MODEL_655003d887be4b0da175663424a146f1"}},"a3bc16f6a15e4fd4b377c59efb7c044c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_054ba05fa0f349c69a8384b0978f298f","placeholder":"​","style":"IPY_MODEL_e56b4ae542e442c48a7363f923fe84e8","value":"Downloading (…)&quot;spiece.model&quot;;: 100%"}},"5efe04cb883e45f0bbdfbbd3e395b15c":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_6cba84a4f97c49b0a070552fc5fbff76","max":791656,"min":0,"orientation":"horizontal","style":"IPY_MODEL_5310dfef2ca940ce936ecd5076bfaf0d","value":791656}},"0dc054f35b374f878a24d535bdfad9fb":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_959c58ebf0c341ecbcb6d6c652ff0113","placeholder":"​","style":"IPY_MODEL_084c1fb97ce1492da72682cfd07dc47d","value":" 792k/792k [00:00&lt;00:00, 8.36MB/s]"}},"655003d887be4b0da175663424a146f1":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"054ba05fa0f349c69a8384b0978f298f":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e56b4ae542e442c48a7363f923fe84e8":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6cba84a4f97c49b0a070552fc5fbff76":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5310dfef2ca940ce936ecd5076bfaf0d":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"959c58ebf0c341ecbcb6d6c652ff0113":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"084c1fb97ce1492da72682cfd07dc47d":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7bd9d152451a4d5eb563b980c99fc211":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_c5e083e45aae422db9e3c2527e3823ce","IPY_MODEL_a8d3b906a8c14700b59cb8a711660288","IPY_MODEL_43f4a197420c46b1840dbfdf9c21266e"],"layout":"IPY_MODEL_b0ff87a5842e473ea003412248fbadc7"}},"c5e083e45aae422db9e3c2527e3823ce":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_cdc97857b95c4d889cc8b000daa87846","placeholder":"​","style":"IPY_MODEL_4bf9cf5009fb43a4bba04c5e42f25da8","value":"Downloading (…)/main/tokenizer.json: 100%"}},"a8d3b906a8c14700b59cb8a711660288":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a50180e15c284cea9950d107463eaaeb","max":2424064,"min":0,"orientation":"horizontal","style":"IPY_MODEL_d6e276e5fc60476783b10775a5e3ef7e","value":2424064}},"43f4a197420c46b1840dbfdf9c21266e":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_88061a9e33c64e1fa65cbdcd7f586564","placeholder":"​","style":"IPY_MODEL_2fc4ac2558194f869bf131192b053273","value":" 2.42M/2.42M [00:00&lt;00:00, 9.87MB/s]"}},"b0ff87a5842e473ea003412248fbadc7":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cdc97857b95c4d889cc8b000daa87846":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4bf9cf5009fb43a4bba04c5e42f25da8":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a50180e15c284cea9950d107463eaaeb":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d6e276e5fc60476783b10775a5e3ef7e":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"88061a9e33c64e1fa65cbdcd7f586564":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2fc4ac2558194f869bf131192b053273":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"902a66286d5b4dffad200da7adf745c7":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_7f7349e82d58494d82c67bcea1a259fd","IPY_MODEL_01c3ab1eda3a4b41a8108dae2d06bf95","IPY_MODEL_cde3998b720b45f68d4c1733e93d6ea6"],"layout":"IPY_MODEL_59900edf25e84470abcd1d3fc4cd0ef5"}},"7f7349e82d58494d82c67bcea1a259fd":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b87577f11a474ef39da02e0b8195524d","placeholder":"​","style":"IPY_MODEL_3df99a08280c4e16961a51acd5a2c701","value":"Downloading (…)cial_tokens_map.json: 100%"}},"01c3ab1eda3a4b41a8108dae2d06bf95":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_3b91340d7d2e45b3bfba0c46b4bf476c","max":2201,"min":0,"orientation":"horizontal","style":"IPY_MODEL_7083f9df7e92477cbf734faa7363c35a","value":2201}},"cde3998b720b45f68d4c1733e93d6ea6":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ceddd92f40d44737acfced85e2118198","placeholder":"​","style":"IPY_MODEL_aa7a986d41594a8e8ffb08793ca56b0a","value":" 2.20k/2.20k [00:00&lt;00:00, 52.7kB/s]"}},"59900edf25e84470abcd1d3fc4cd0ef5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b87577f11a474ef39da02e0b8195524d":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3df99a08280c4e16961a51acd5a2c701":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3b91340d7d2e45b3bfba0c46b4bf476c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7083f9df7e92477cbf734faa7363c35a":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"ceddd92f40d44737acfced85e2118198":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"aa7a986d41594a8e8ffb08793ca56b0a":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9a6fea9adcaf4099ab6c0860aca91af1":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_c98679bf3f404645ba349c9040bcaf3d","IPY_MODEL_59449a1ef9b840b581757da9342f64c4","IPY_MODEL_a7a1bc5e99834919a68f30b1fff34ba7"],"layout":"IPY_MODEL_0c7e07a6e45b42e486c50df3665e3000"}},"c98679bf3f404645ba349c9040bcaf3d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_eea6649f7b4647b8b30a2e9dcb2cc0b8","placeholder":"​","style":"IPY_MODEL_ed3f60219b4148e7b45bdf31f11f4617","value":"Downloading (…)lve/main/config.json: 100%"}},"59449a1ef9b840b581757da9342f64c4":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_89157fabea4c4a0ea563d9e60f7cfa54","max":1404,"min":0,"orientation":"horizontal","style":"IPY_MODEL_b08aaa725b7445a28894a8f5f82c599b","value":1404}},"a7a1bc5e99834919a68f30b1fff34ba7":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_01a7431d2ed440c2875caa50288d36fd","placeholder":"​","style":"IPY_MODEL_f062075516064894ad208522903485ab","value":" 1.40k/1.40k [00:00&lt;00:00, 22.0kB/s]"}},"0c7e07a6e45b42e486c50df3665e3000":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"eea6649f7b4647b8b30a2e9dcb2cc0b8":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ed3f60219b4148e7b45bdf31f11f4617":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"89157fabea4c4a0ea563d9e60f7cfa54":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b08aaa725b7445a28894a8f5f82c599b":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"01a7431d2ed440c2875caa50288d36fd":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f062075516064894ad208522903485ab":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"022661137b474760b175a0b0ddeb844c":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_814cd3e8fc0441d8857d15976e7b0d0d","IPY_MODEL_90e31b2327a34aecb4e483b3a724a721","IPY_MODEL_db9b101ac0e34e659d4cb59a0721fb7d"],"layout":"IPY_MODEL_f0ec79de51c547c789fb9cbd6ede9bc0"}},"814cd3e8fc0441d8857d15976e7b0d0d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_88d8c48bf9ef47c49cbabb12bec227f7","placeholder":"​","style":"IPY_MODEL_8fe33fb112614f1181fb9a9a5f92cf50","value":"Downloading (…)&quot;pytorch_model.bin&quot;;: 100%"}},"90e31b2327a34aecb4e483b3a724a721":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_60a0e9d6ddd24a6a81742db75543d52c","max":990402637,"min":0,"orientation":"horizontal","style":"IPY_MODEL_ed1fc00a86bd4eb0b19b4df04fc01c05","value":990402637}},"db9b101ac0e34e659d4cb59a0721fb7d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0f935ce070ec40e983d97b0b5245e128","placeholder":"​","style":"IPY_MODEL_8f0ac859443a4098ab96f37bacb232b1","value":" 990M/990M [00:09&lt;00:00, 87.9MB/s]"}},"f0ec79de51c547c789fb9cbd6ede9bc0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"88d8c48bf9ef47c49cbabb12bec227f7":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8fe33fb112614f1181fb9a9a5f92cf50":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"60a0e9d6ddd24a6a81742db75543d52c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ed1fc00a86bd4eb0b19b4df04fc01c05":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"0f935ce070ec40e983d97b0b5245e128":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8f0ac859443a4098ab96f37bacb232b1":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"817d648b4b2746068c566e92cd07601d":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_1bd7cb6802654dc9a8545b44b1cd74dc","IPY_MODEL_8d6609cdcb554d59b7e2a74fbd8272b3","IPY_MODEL_8e6a5f8ef175462a9d4570631ab46232"],"layout":"IPY_MODEL_ba87e1b9ed8149f3b0ddfdc63ac1dd83"}},"1bd7cb6802654dc9a8545b44b1cd74dc":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9ba43ff9c97f458d9eea292e0b04bb72","placeholder":"​","style":"IPY_MODEL_cc48f3926f63490ea63cc74e4df6815c","value":"Downloading (…)neration_config.json: 100%"}},"8d6609cdcb554d59b7e2a74fbd8272b3":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_612651def7564e56949cde5f4a340f2c","max":147,"min":0,"orientation":"horizontal","style":"IPY_MODEL_783b209f9a58455288153f267aae517b","value":147}},"8e6a5f8ef175462a9d4570631ab46232":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3c90af09603e4f9b8d1d9730f8d3660c","placeholder":"​","style":"IPY_MODEL_0d98d0018cfb4cfab57b9ef7ff5ac394","value":" 147/147 [00:00&lt;00:00, 4.87kB/s]"}},"ba87e1b9ed8149f3b0ddfdc63ac1dd83":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9ba43ff9c97f458d9eea292e0b04bb72":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cc48f3926f63490ea63cc74e4df6815c":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"612651def7564e56949cde5f4a340f2c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"783b209f9a58455288153f267aae517b":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"3c90af09603e4f9b8d1d9730f8d3660c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0d98d0018cfb4cfab57b9ef7ff5ac394":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","source":["# Instalacja pakietów"],"metadata":{"id":"ZXsOR6oJOJbd"}},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"8l0hzptKNiZS","executionInfo":{"status":"ok","timestamp":1676655847146,"user_tz":-60,"elapsed":22906,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"outputId":"8c0d45c5-f6e8-4db2-b8bb-24c258e6f19e"},"outputs":[{"output_type":"stream","name":"stdout","text":["Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n","Collecting transformers\n","  Downloading transformers-4.26.1-py3-none-any.whl (6.3 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.3/6.3 MB\u001b[0m \u001b[31m42.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting datasets\n","  Downloading datasets-2.9.0-py3-none-any.whl (462 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m462.8/462.8 KB\u001b[0m \u001b[31m34.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: torch in /usr/local/lib/python3.8/dist-packages (1.13.1+cu116)\n","Collecting sentencepiece\n","  Downloading sentencepiece-0.1.97-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m49.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting huggingface-hub<1.0,>=0.11.0\n","  Downloading huggingface_hub-0.12.1-py3-none-any.whl (190 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.3/190.3 KB\u001b[0m \u001b[31m15.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting tokenizers!=0.11.3,<0.14,>=0.11.1\n","  Downloading tokenizers-0.13.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.6 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.6/7.6 MB\u001b[0m \u001b[31m23.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (1.21.6)\n","Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.8/dist-packages (from transformers) (4.64.1)\n","Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (2022.6.2)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers) (3.9.0)\n","Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (23.0)\n","Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from transformers) (2.25.1)\n","Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (6.0)\n","Requirement already satisfied: pyarrow>=6.0.0 in /usr/local/lib/python3.8/dist-packages (from datasets) (9.0.0)\n","Collecting xxhash\n","  Downloading xxhash-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (213 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m213.0/213.0 KB\u001b[0m \u001b[31m6.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting multiprocess\n","  Downloading multiprocess-0.70.14-py38-none-any.whl (132 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m132.0/132.0 KB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: aiohttp in /usr/local/lib/python3.8/dist-packages (from datasets) (3.8.4)\n","Requirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.8/dist-packages (from datasets) (2023.1.0)\n","Requirement already satisfied: dill<0.3.7 in /usr/local/lib/python3.8/dist-packages (from datasets) (0.3.6)\n","Collecting responses<0.19\n","  Downloading responses-0.18.0-py3-none-any.whl (38 kB)\n","Requirement already satisfied: pandas in /usr/local/lib/python3.8/dist-packages (from datasets) (1.3.5)\n","Requirement already satisfied: typing-extensions in /usr/local/lib/python3.8/dist-packages (from torch) (4.5.0)\n","Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (4.0.2)\n","Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.8.2)\n","Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.1)\n","Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (22.2.0)\n","Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (3.0.1)\n","Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.3)\n","Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (6.0.4)\n","Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (4.0.0)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2022.12.7)\n","Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (1.24.3)\n","Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2.10)\n","Collecting urllib3<1.27,>=1.21.1\n","  Downloading urllib3-1.26.14-py2.py3-none-any.whl (140 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m140.6/140.6 KB\u001b[0m \u001b[31m11.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas->datasets) (2.8.2)\n","Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas->datasets) (2022.7.1)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil>=2.7.3->pandas->datasets) (1.15.0)\n","Installing collected packages: tokenizers, sentencepiece, xxhash, urllib3, multiprocess, responses, huggingface-hub, transformers, datasets\n","  Attempting uninstall: urllib3\n","    Found existing installation: urllib3 1.24.3\n","    Uninstalling urllib3-1.24.3:\n","      Successfully uninstalled urllib3-1.24.3\n","Successfully installed datasets-2.9.0 huggingface-hub-0.12.1 multiprocess-0.70.14 responses-0.18.0 sentencepiece-0.1.97 tokenizers-0.13.2 transformers-4.26.1 urllib3-1.26.14 xxhash-3.2.0\n"]}],"source":["!pip install transformers datasets torch sentencepiece"]},{"cell_type":"markdown","source":["# Załadowanie datasetu"],"metadata":{"id":"dhN0rmb5Oi3d"}},{"cell_type":"code","source":["from datasets import load_dataset"],"metadata":{"id":"tnaDkwZ2Pbnn","executionInfo":{"status":"ok","timestamp":1676655848707,"user_tz":-60,"elapsed":1224,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}}},"execution_count":2,"outputs":[]},{"cell_type":"code","source":["dataset = load_dataset(\"sms_spam\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":232,"referenced_widgets":["25db59aa53d542bfb6d54da340006fcd","2eaefa22c2e14c40a2de9592676a8d82","1bf81e50052942208c5100bfdf0c985a","5422e114814145d1a85b423b93d4133c","0031c9a8c8724bce9aff6b3cafc94c2b","3d3b84701cbf468c96170f7563e42408","96bdc942fcca434798c89e4ec74b6dce","5cabe61f114a4c33ae677d9b5306c635","6b39afc3b8bc43768b376186aafe96de","7bd83463f6504145b0ca14ab78750d3a","61d94df2723e4e2d94cd74a1b884d409","d160c37c8eb241eda89f7713fd4d05c2","5fa8d14ffdbc45e6812b50e0d27e14fe","a54fa4aedddc46ba84224971eb1e5227","98c7b45870cc49b7bb3addd225e04301","6f3ada40efe54c13b2c14ad60a6b18e4","f82774bb70cc4f90b80d723f8aa9c7e4","b3f23480d62d4c50a8aaa03118f6b5ce","3a9cfb628ef04d8ca222584f40be05c4","9bdb6d0c6fa54454957211029a960159","67ab4e3216154b67bcdac51d4548ed75","524913d4b3ca4f729acfc975ea490b46","a543b6e11ab046748cdd6c946a1f9084","8c2521b501c64c90998bc05d1a76281c","bca85efcfcd546b9a43ce79a68b7f019","0f359da3c8b1402b8e391c2360531d19","1a6b59d6f8e54a30b69822e6b59fd99f","dccb6a9167aa4b60898047d46363d80e","545f9b3e11cf42818c25709159e7fb3e","6af88f49ff5346b9a0ad07faceede0bd","fc87784596044dd78d045989dbb53a91","331e1896aa424786a810a711143d478e","a39965de69714b26a78f8b976a33a42e","80a4187b45134a4ab30a7748e4f6ff28","326f7ae8ea614699a30da8dfe3f929be","c7574a234b8d492bba0fec988d8e8ae1","2282de3148204153873e6c678d3ab97d","9a9392cc5903455e9206d9c965f5005c","0c48c929e21f4858b9323b7f4ac220b2","17dd8779c5d642ceaa85e0d7c8524436","e5e5197e03c040a99a5d31c57f8a047c","a9aa4d15ad5143b1a5686972ef47186e","d23808437da04c37868bd7526e685a5c","6ac94e8230564af9b05ac561ab5d8e7b","4f03235725824725add5a46e03a5156e","af2ac44def004870a388284503a45b07","3a70f4da67754c5796c8ab5e8eaec187","1c0621a41647430d8dfda34f2f7d8e2f","d0d7c235a53e46e98f99229f26b665d0","accb4a02861c4ad3a7965ba707ab8a38","7d7810f28a2342848026eb36e967d8a5","f7f34fc698bc416383ae59828301469d","373bf37aada844a187f13af6d723333b","7f309cfb6bc54a5e8872524eb63354c2","c69df495e9324100b87aae91c9b34bf6","209b3c2d088b4512bf513e17e74e8a30","eef55327290e4e98b5ac4b2c3e8b4ad4","495b6416c57149bfa9d8418676c4541d","4c01dbed470a4fbc94aaedc3c679385f","dda3f23953f64ceead909262eb0f629d","6165a4a6a5454cd8b73c9702771dcae1","ff4d3372dcc14403aeb19ff960a518f5","dea72bac03c1492da956598f66465472","4b87d5b0da3c4a6484d1757a09de382b","fe6080acb5544dd8a3c512fbf38afd87","1b4bf9de6c59492091f84d80871545c5"]},"id":"cCiAuRqrOkvV","executionInfo":{"status":"ok","timestamp":1676655851010,"user_tz":-60,"elapsed":2305,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"outputId":"78480a8f-563d-42a5-c5dd-4bdfa1f8e78b"},"execution_count":3,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading builder script:   0%|          | 0.00/3.21k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"25db59aa53d542bfb6d54da340006fcd"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading metadata:   0%|          | 0.00/1.69k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"d160c37c8eb241eda89f7713fd4d05c2"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading readme:   0%|          | 0.00/4.87k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"a543b6e11ab046748cdd6c946a1f9084"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Downloading and preparing dataset sms_spam/plain_text to /root/.cache/huggingface/datasets/sms_spam/plain_text/1.0.0/53f051d3b5f62d99d61792c91acefe4f1577ad3e4c216fb0ad39e30b9f20019c...\n"]},{"output_type":"display_data","data":{"text/plain":["Downloading data:   0%|          | 0.00/203k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"80a4187b45134a4ab30a7748e4f6ff28"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Generating train split:   0%|          | 0/5574 [00:00<?, ? examples/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"4f03235725824725add5a46e03a5156e"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Dataset sms_spam downloaded and prepared to /root/.cache/huggingface/datasets/sms_spam/plain_text/1.0.0/53f051d3b5f62d99d61792c91acefe4f1577ad3e4c216fb0ad39e30b9f20019c. Subsequent calls will reuse this data.\n"]},{"output_type":"display_data","data":{"text/plain":["  0%|          | 0/1 [00:00<?, ?it/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"209b3c2d088b4512bf513e17e74e8a30"}},"metadata":{}}]},{"cell_type":"code","source":["dataset['train'][0]"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"JKFHPko3OnAV","executionInfo":{"status":"ok","timestamp":1676655851011,"user_tz":-60,"elapsed":7,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"outputId":"f813fa0d-c053-4b6e-a5f6-f304b5447912"},"execution_count":4,"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\\n',\n"," 'label': 0}"]},"metadata":{},"execution_count":4}]},{"cell_type":"markdown","source":["# Przygotowanie datasetu"],"metadata":{"id":"l140vJrgYxPr"}},{"cell_type":"code","source":["parsed_dataset = []\n","\n","for row in dataset['train']:\n","  text = row['sms'].replace(\"\\n\", \"\")\n","  new_row = {}\n","  new_row['sms'] = text\n","  if row['label'] == 0:\n","    new_row['label'] = \"False\"\n","  else:\n","    new_row['label'] = \"True\"\n","  parsed_dataset.append(new_row)\n","\n","parsed_dataset[0]"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"1boUF-YiY3_y","executionInfo":{"status":"ok","timestamp":1676655851377,"user_tz":-60,"elapsed":371,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"outputId":"1435212c-73cb-4660-d9ec-e06d7fe5e493"},"execution_count":5,"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...',\n"," 'label': 'False'}"]},"metadata":{},"execution_count":5}]},{"cell_type":"markdown","source":["# Tokenizer FLAN-T5"],"metadata":{"id":"O-J-jBDxPJcn"}},{"cell_type":"code","source":["from transformers import AutoTokenizer"],"metadata":{"id":"P23AYPX1PZ6g","executionInfo":{"status":"ok","timestamp":1676655853434,"user_tz":-60,"elapsed":2058,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}}},"execution_count":6,"outputs":[]},{"cell_type":"code","source":["tokenizer = AutoTokenizer.from_pretrained('google/flan-t5-base')"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":145,"referenced_widgets":["d5c9c4fcc72f46639246e0e733af5ff8","8b8719c742754a15baa3c14a9f2f1852","b309af1852fc4d8d8985f2690b768485","30237ebd73554b428d13a66f18d90069","e80a747c0d934144aa2b78cff101c87a","494c694bd92e49d3831d716cb90b4dd8","e03d7f43bdbc4deda2c35eaaece29fe2","56aa37b752a54936a09e2167f0d2c281","492a083bfe674cd496a91fc8c51a4646","58865aaf52a94747b20aac75ebb46426","6a609ebaee3b465fa35107769eb4cc9d","6a23888d5c8a4080b02e6b632f14c6ef","a3bc16f6a15e4fd4b377c59efb7c044c","5efe04cb883e45f0bbdfbbd3e395b15c","0dc054f35b374f878a24d535bdfad9fb","655003d887be4b0da175663424a146f1","054ba05fa0f349c69a8384b0978f298f","e56b4ae542e442c48a7363f923fe84e8","6cba84a4f97c49b0a070552fc5fbff76","5310dfef2ca940ce936ecd5076bfaf0d","959c58ebf0c341ecbcb6d6c652ff0113","084c1fb97ce1492da72682cfd07dc47d","7bd9d152451a4d5eb563b980c99fc211","c5e083e45aae422db9e3c2527e3823ce","a8d3b906a8c14700b59cb8a711660288","43f4a197420c46b1840dbfdf9c21266e","b0ff87a5842e473ea003412248fbadc7","cdc97857b95c4d889cc8b000daa87846","4bf9cf5009fb43a4bba04c5e42f25da8","a50180e15c284cea9950d107463eaaeb","d6e276e5fc60476783b10775a5e3ef7e","88061a9e33c64e1fa65cbdcd7f586564","2fc4ac2558194f869bf131192b053273","902a66286d5b4dffad200da7adf745c7","7f7349e82d58494d82c67bcea1a259fd","01c3ab1eda3a4b41a8108dae2d06bf95","cde3998b720b45f68d4c1733e93d6ea6","59900edf25e84470abcd1d3fc4cd0ef5","b87577f11a474ef39da02e0b8195524d","3df99a08280c4e16961a51acd5a2c701","3b91340d7d2e45b3bfba0c46b4bf476c","7083f9df7e92477cbf734faa7363c35a","ceddd92f40d44737acfced85e2118198","aa7a986d41594a8e8ffb08793ca56b0a"]},"id":"q5Jz0E_oPMBr","executionInfo":{"status":"ok","timestamp":1676655855532,"user_tz":-60,"elapsed":2100,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"outputId":"d83aebe2-e6ae-463c-b189-140b429379bd"},"execution_count":7,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading (…)okenizer_config.json:   0%|          | 0.00/2.54k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"d5c9c4fcc72f46639246e0e733af5ff8"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)\"spiece.model\";:   0%|          | 0.00/792k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6a23888d5c8a4080b02e6b632f14c6ef"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)/main/tokenizer.json:   0%|          | 0.00/2.42M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"7bd9d152451a4d5eb563b980c99fc211"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)cial_tokens_map.json:   0%|          | 0.00/2.20k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"902a66286d5b4dffad200da7adf745c7"}},"metadata":{}}]},{"cell_type":"code","source":["sms = parsed_dataset[0]['sms']\n","print('Original: ', sms)\n","print('Tokenized: ', tokenizer.tokenize(sms))\n","print('Token IDs: ', tokenizer.convert_tokens_to_ids(tokenizer.tokenize(sms)))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"dfxJQpoePsvI","executionInfo":{"status":"ok","timestamp":1676655855533,"user_tz":-60,"elapsed":13,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"outputId":"c3542a4d-1bbf-4579-e6c3-9641b9802201"},"execution_count":8,"outputs":[{"output_type":"stream","name":"stdout","text":["Original:  Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\n","Tokenized:  ['▁Go', '▁until', '▁jur', 'ong', '▁point', ',', '▁crazy', '.', '.', '▁Available', '▁only', '▁in', '▁bug', 'is', '▁', 'n', '▁great', '▁world', '▁la', '▁', 'e', '▁buffet', '...', '▁Cine', '▁there', '▁got', '▁', 'a', 'more', '▁wa', 't', '...']\n","Token IDs:  [1263, 552, 10081, 2444, 500, 6, 6139, 5, 5, 8144, 163, 16, 8143, 159, 3, 29, 248, 296, 50, 3, 15, 15385, 233, 17270, 132, 530, 3, 9, 3706, 8036, 17, 233]\n"]}]},{"cell_type":"markdown","source":["# Few shot learning"],"metadata":{"id":"UpluhM8cU5Ir"}},{"cell_type":"code","source":["print(parsed_dataset[0]) #0\n","print(parsed_dataset[123]) #1\n","print(parsed_dataset[2000]) #0\n","print(parsed_dataset[3002]) #1"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"7uNUkixPU85O","executionInfo":{"status":"ok","timestamp":1676655855534,"user_tz":-60,"elapsed":11,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"outputId":"bfd36343-8751-4525-d745-c4612ecae5bc"},"execution_count":9,"outputs":[{"output_type":"stream","name":"stdout","text":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...', 'label': 'False'}\n","{'sms': 'Todays Voda numbers ending 7548 are selected to receive a $350 award. If you have a match please call 08712300220 quoting claim code 4041 standard rates app', 'label': 'True'}\n","{'sms': \"LMAO where's your fish memory when I need it?\", 'label': 'False'}\n","{'sms': 'This message is free. Welcome to the new & improved Sex & Dogging club! To unsubscribe from this service reply STOP. msgs@150p 18+only', 'label': 'True'}\n"]}]},{"cell_type":"code","source":["non_spam_1 = \"The text message contains the following content: \" + parsed_dataset[0]['sms'] + \"\\nIs it true that the content of a text message is spam: False\\n\\n\"\n","spam_1 = \"The text message contains the following content: \" + parsed_dataset[123]['sms'] + \"\\nIs it true that the content of a text message is spam: True\\n\\n\"\n","non_spam_2 = \"The text message contains the following content: \" + parsed_dataset[2000]['sms'] + \"\\nIs it true that the content of a text message is spam: False\\n\\n\"\n","spam_2 = \"The text message contains the following content: \" + parsed_dataset[3002]['sms'] + \"\\nIs it true that the content of a text message is spam: True\\n\\n\"\n","\n","few_shot_prefix = non_spam_1 + spam_1 + non_spam_2 + spam_2 + \"The text message contains the following content: \"\n","print(few_shot_prefix)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"lj0issBznZfK","executionInfo":{"status":"ok","timestamp":1676655855534,"user_tz":-60,"elapsed":8,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"outputId":"b808dc44-b404-4c61-a4ca-c225961b22ac"},"execution_count":10,"outputs":[{"output_type":"stream","name":"stdout","text":["The text message contains the following content: Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\n","Is it true that the content of a text message is spam: False\n","\n","The text message contains the following content: Todays Voda numbers ending 7548 are selected to receive a $350 award. If you have a match please call 08712300220 quoting claim code 4041 standard rates app\n","Is it true that the content of a text message is spam: True\n","\n","The text message contains the following content: LMAO where's your fish memory when I need it?\n","Is it true that the content of a text message is spam: False\n","\n","The text message contains the following content: This message is free. Welcome to the new & improved Sex & Dogging club! To unsubscribe from this service reply STOP. msgs@150p 18+only\n","Is it true that the content of a text message is spam: True\n","\n","The text message contains the following content: \n"]}]},{"cell_type":"markdown","source":["# Load FLAN-T5 model"],"metadata":{"id":"okTx_ynMV0rH"}},{"cell_type":"code","source":["from transformers import AutoModelForSeq2SeqLM"],"metadata":{"id":"Eu-7Eed8WgN0","executionInfo":{"status":"ok","timestamp":1676655855535,"user_tz":-60,"elapsed":6,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}}},"execution_count":11,"outputs":[]},{"cell_type":"code","source":["model = AutoModelForSeq2SeqLM.from_pretrained('google/flan-t5-base')\n","\n","model.cuda()"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000,"referenced_widgets":["9a6fea9adcaf4099ab6c0860aca91af1","c98679bf3f404645ba349c9040bcaf3d","59449a1ef9b840b581757da9342f64c4","a7a1bc5e99834919a68f30b1fff34ba7","0c7e07a6e45b42e486c50df3665e3000","eea6649f7b4647b8b30a2e9dcb2cc0b8","ed3f60219b4148e7b45bdf31f11f4617","89157fabea4c4a0ea563d9e60f7cfa54","b08aaa725b7445a28894a8f5f82c599b","01a7431d2ed440c2875caa50288d36fd","f062075516064894ad208522903485ab","022661137b474760b175a0b0ddeb844c","814cd3e8fc0441d8857d15976e7b0d0d","90e31b2327a34aecb4e483b3a724a721","db9b101ac0e34e659d4cb59a0721fb7d","f0ec79de51c547c789fb9cbd6ede9bc0","88d8c48bf9ef47c49cbabb12bec227f7","8fe33fb112614f1181fb9a9a5f92cf50","60a0e9d6ddd24a6a81742db75543d52c","ed1fc00a86bd4eb0b19b4df04fc01c05","0f935ce070ec40e983d97b0b5245e128","8f0ac859443a4098ab96f37bacb232b1","817d648b4b2746068c566e92cd07601d","1bd7cb6802654dc9a8545b44b1cd74dc","8d6609cdcb554d59b7e2a74fbd8272b3","8e6a5f8ef175462a9d4570631ab46232","ba87e1b9ed8149f3b0ddfdc63ac1dd83","9ba43ff9c97f458d9eea292e0b04bb72","cc48f3926f63490ea63cc74e4df6815c","612651def7564e56949cde5f4a340f2c","783b209f9a58455288153f267aae517b","3c90af09603e4f9b8d1d9730f8d3660c","0d98d0018cfb4cfab57b9ef7ff5ac394"]},"id":"JKv9O8kfV2zZ","executionInfo":{"status":"ok","timestamp":1676655876634,"user_tz":-60,"elapsed":21105,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"outputId":"785dda59-be66-4f68-88e2-0ff978a4d7e7"},"execution_count":12,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading (…)lve/main/config.json:   0%|          | 0.00/1.40k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"9a6fea9adcaf4099ab6c0860aca91af1"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)\"pytorch_model.bin\";:   0%|          | 0.00/990M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"022661137b474760b175a0b0ddeb844c"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)neration_config.json:   0%|          | 0.00/147 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"817d648b4b2746068c566e92cd07601d"}},"metadata":{}},{"output_type":"execute_result","data":{"text/plain":["T5ForConditionalGeneration(\n","  (shared): Embedding(32128, 768)\n","  (encoder): T5Stack(\n","    (embed_tokens): Embedding(32128, 768)\n","    (block): ModuleList(\n","      (0): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","              (relative_attention_bias): Embedding(32, 12)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (1): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (2): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (3): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (4): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (5): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (6): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (7): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (8): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (9): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (10): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (11): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","    )\n","    (final_layer_norm): T5LayerNorm()\n","    (dropout): Dropout(p=0.1, inplace=False)\n","  )\n","  (decoder): T5Stack(\n","    (embed_tokens): Embedding(32128, 768)\n","    (block): ModuleList(\n","      (0): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","              (relative_attention_bias): Embedding(32, 12)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (1): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (2): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (3): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (4): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (5): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (6): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (7): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (8): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (9): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (10): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (11): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","    )\n","    (final_layer_norm): T5LayerNorm()\n","    (dropout): Dropout(p=0.1, inplace=False)\n","  )\n","  (lm_head): Linear(in_features=768, out_features=32128, bias=False)\n",")"]},"metadata":{},"execution_count":12}]},{"cell_type":"markdown","source":["# Helper functions"],"metadata":{"id":"F_SDAwxoawDy"}},{"cell_type":"code","source":["import torch"],"metadata":{"id":"rdWMg_KJZEZH","executionInfo":{"status":"ok","timestamp":1676655876634,"user_tz":-60,"elapsed":16,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}}},"execution_count":13,"outputs":[]},{"cell_type":"code","source":["def calculate_accuracy(preds, target):\n","  results_ok = 0.0\n","  results_false = 0.0\n","\n","  for idx, pred in enumerate(preds):\n","    if pred == target[idx]:\n","      results_ok += 1.0\n","    else:\n","      results_false += 1.0\n","\n","  return results_ok / (results_ok + results_false)"],"metadata":{"id":"FzUi8908ax61","executionInfo":{"status":"ok","timestamp":1676655876634,"user_tz":-60,"elapsed":15,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}}},"execution_count":14,"outputs":[]},{"cell_type":"code","source":["if torch.cuda.is_available():     \n","    device = torch.device(\"cuda\")\n","\n","    print('There are %d GPU(s) available.' % torch.cuda.device_count())\n","    print('We will use the GPU:', torch.cuda.get_device_name(0))\n","\n","else:\n","    print('No GPU available, using the CPU instead.')\n","    device = torch.device(\"cpu\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"86i7iRmtW-6L","executionInfo":{"status":"ok","timestamp":1676655876635,"user_tz":-60,"elapsed":15,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"outputId":"c3e396e1-5ba3-42bc-9fb8-d22bb6052bdc"},"execution_count":15,"outputs":[{"output_type":"stream","name":"stdout","text":["There are 1 GPU(s) available.\n","We will use the GPU: Tesla T4\n"]}]},{"cell_type":"markdown","source":["# Predykcja"],"metadata":{"id":"H_YI3bS3VHQE"}},{"cell_type":"code","source":["parsed_dataset = parsed_dataset[1:123] + parsed_dataset[124:2000] + parsed_dataset[2001:3002] + parsed_dataset[3003:]\n","predictions = []\n","expected = []\n","\n","for row in parsed_dataset:\n","  input_text = few_shot_prefix + row['sms'] + \"\\nIs it true that the content of a text message is spam: \"\n","  input_ids = tokenizer(input_text, return_tensors=\"pt\").input_ids.to(device)\n","\n","  generated_ids = model.generate(input_ids, do_sample=True, temperature=0.9, max_length=250)\n","  generated_text = tokenizer.decode(generated_ids[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)\n","  \n","  predictions.append(generated_text)\n","  expected.append(row['label'])\n","\n","acc = calculate_accuracy(predictions, expected)\n","print(acc)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"vunLGuBGVGmh","executionInfo":{"status":"ok","timestamp":1676656668059,"user_tz":-60,"elapsed":484351,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"outputId":"fe21ecec-0bee-4730-9b15-0a86aa695dec"},"execution_count":17,"outputs":[{"output_type":"stream","name":"stdout","text":["0.2551203736974488\n"]}]},{"cell_type":"code","source":["print(\"Sample prediction: {}, expected: {}\".format(predictions[101], expected[101]))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"20WbYZmLaDl7","executionInfo":{"status":"ok","timestamp":1676657297829,"user_tz":-60,"elapsed":315,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"outputId":"ea897a66-802e-4bdc-93b5-f9f81f2129dc"},"execution_count":18,"outputs":[{"output_type":"stream","name":"stdout","text":["Sample prediction: No, expected: False\n"]}]},{"cell_type":"markdown","source":["# MCC Score"],"metadata":{"id":"dLYc9WXz_B1o"}},{"cell_type":"code","source":["from sklearn.metrics import matthews_corrcoef\n","\n","print('Calculating Matthews Corr. Coef. for each batch...')\n","matthews = matthews_corrcoef(expected, predictions)    \n","print('Total MCC: %.3f' % matthews)"],"metadata":{"id":"hPEPpXXX_DXR","executionInfo":{"status":"ok","timestamp":1676657301430,"user_tz":-60,"elapsed":1271,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"79633d5e-37ca-4374-874b-f4ba75f238b0"},"execution_count":19,"outputs":[{"output_type":"stream","name":"stdout","text":["Calculating Matthews Corr. Coef. for each batch...\n","Total MCC: -0.020\n"]}]},{"cell_type":"markdown","source":["# Save model"],"metadata":{"id":"GPhCp068_Iwq"}},{"cell_type":"code","source":["from google.colab import drive\n","\n","drive.mount('/content/gdrive/', force_remount=True)\n","\n","output_dir = '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_3_FLAN-T5_Model'\n","print(\"Saving model to %s\" % output_dir)\n","\n","model_to_save = model.module if hasattr(model, 'module') else model\n","model_to_save.save_pretrained(output_dir)\n","tokenizer.save_pretrained(output_dir)"],"metadata":{"id":"avafCMoS_KDF","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1676657344651,"user_tz":-60,"elapsed":38045,"user":{"displayName":"Mike1 Studia1","userId":"14831644302217245682"}},"outputId":"39895ea5-7683-4df5-fd99-5a67ce5cc2e8"},"execution_count":20,"outputs":[{"output_type":"stream","name":"stdout","text":["Mounted at /content/gdrive/\n","Saving model to /content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_3_FLAN-T5_Model\n"]},{"output_type":"execute_result","data":{"text/plain":["('/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_3_FLAN-T5_Model/tokenizer_config.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_3_FLAN-T5_Model/special_tokens_map.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_3_FLAN-T5_Model/spiece.model',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_3_FLAN-T5_Model/added_tokens.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_3_FLAN-T5_Model/tokenizer.json')"]},"metadata":{},"execution_count":20}]},{"cell_type":"markdown","source":["# Bibliografia\n","- https://huggingface.co/docs/transformers/main/en/model_doc/flan-t5\n","- https://mccormickml.com/2019/07/22/BERT-fine-tuning/#a1-saving--loading-fine-tuned-model\n","- https://huggingface.co/docs/transformers/model_doc/t5#training"],"metadata":{"id":"wHzm2_nDA6i-"}}]}
\ No newline at end of file
diff --git a/projekt/Ver_2_FLAN-T5_sms_spam.ipynb b/projekt/Ver_2_FLAN-T5_sms_spam.ipynb
new file mode 100644
index 0000000..7f5011e
--- /dev/null
+++ b/projekt/Ver_2_FLAN-T5_sms_spam.ipynb
@@ -0,0 +1 @@
+{"cells":[{"cell_type":"markdown","metadata":{"id":"ZXsOR6oJOJbd"},"source":["# Instalacja pakietów"]},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":23167,"status":"ok","timestamp":1676653734837,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"8l0hzptKNiZS","outputId":"2155e4e2-158a-43be-acb9-2c78176c20b1"},"outputs":[{"name":"stdout","output_type":"stream","text":["Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n","Collecting transformers\n","  Downloading transformers-4.26.1-py3-none-any.whl (6.3 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.3/6.3 MB\u001b[0m \u001b[31m58.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting datasets\n","  Downloading datasets-2.9.0-py3-none-any.whl (462 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m462.8/462.8 KB\u001b[0m \u001b[31m22.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: torch in /usr/local/lib/python3.8/dist-packages (1.13.1+cu116)\n","Collecting sentencepiece\n","  Downloading sentencepiece-0.1.97-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m49.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from transformers) (2.25.1)\n","Requirement already satisfied: packaging\u003e=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (23.0)\n","Requirement already satisfied: tqdm\u003e=4.27 in /usr/local/lib/python3.8/dist-packages (from transformers) (4.64.1)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers) (3.9.0)\n","Requirement already satisfied: pyyaml\u003e=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (6.0)\n","Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (2022.6.2)\n","Requirement already satisfied: numpy\u003e=1.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (1.21.6)\n","Collecting tokenizers!=0.11.3,\u003c0.14,\u003e=0.11.1\n","  Downloading tokenizers-0.13.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.6 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.6/7.6 MB\u001b[0m \u001b[31m42.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting huggingface-hub\u003c1.0,\u003e=0.11.0\n","  Downloading huggingface_hub-0.12.1-py3-none-any.whl (190 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.3/190.3 KB\u001b[0m \u001b[31m13.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.8/dist-packages (from datasets) (1.3.5)\n","Requirement already satisfied: dill\u003c0.3.7 in /usr/local/lib/python3.8/dist-packages (from datasets) (0.3.6)\n","Collecting multiprocess\n","  Downloading multiprocess-0.70.14-py38-none-any.whl (132 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m132.0/132.0 KB\u001b[0m \u001b[31m2.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting responses\u003c0.19\n","  Downloading responses-0.18.0-py3-none-any.whl (38 kB)\n","Requirement already satisfied: fsspec[http]\u003e=2021.11.1 in /usr/local/lib/python3.8/dist-packages (from datasets) (2023.1.0)\n","Requirement already satisfied: pyarrow\u003e=6.0.0 in /usr/local/lib/python3.8/dist-packages (from datasets) (9.0.0)\n","Collecting xxhash\n","  Downloading xxhash-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (213 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m213.0/213.0 KB\u001b[0m \u001b[31m7.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: aiohttp in /usr/local/lib/python3.8/dist-packages (from datasets) (3.8.3)\n","Requirement already satisfied: typing-extensions in /usr/local/lib/python3.8/dist-packages (from torch) (4.4.0)\n","Requirement already satisfied: charset-normalizer\u003c3.0,\u003e=2.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp-\u003edatasets) (2.1.1)\n","Requirement already satisfied: async-timeout\u003c5.0,\u003e=4.0.0a3 in /usr/local/lib/python3.8/dist-packages (from aiohttp-\u003edatasets) (4.0.2)\n","Requirement already satisfied: yarl\u003c2.0,\u003e=1.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp-\u003edatasets) (1.8.2)\n","Requirement already satisfied: aiosignal\u003e=1.1.2 in /usr/local/lib/python3.8/dist-packages (from aiohttp-\u003edatasets) (1.3.1)\n","Requirement already satisfied: attrs\u003e=17.3.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp-\u003edatasets) (22.2.0)\n","Requirement already satisfied: multidict\u003c7.0,\u003e=4.5 in /usr/local/lib/python3.8/dist-packages (from aiohttp-\u003edatasets) (6.0.4)\n","Requirement already satisfied: frozenlist\u003e=1.1.1 in /usr/local/lib/python3.8/dist-packages (from aiohttp-\u003edatasets) (1.3.3)\n","Requirement already satisfied: idna\u003c3,\u003e=2.5 in /usr/local/lib/python3.8/dist-packages (from requests-\u003etransformers) (2.10)\n","Requirement already satisfied: urllib3\u003c1.27,\u003e=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests-\u003etransformers) (1.24.3)\n","Requirement already satisfied: chardet\u003c5,\u003e=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests-\u003etransformers) (4.0.0)\n","Requirement already satisfied: certifi\u003e=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests-\u003etransformers) (2022.12.7)\n","Collecting urllib3\u003c1.27,\u003e=1.21.1\n","  Downloading urllib3-1.26.14-py2.py3-none-any.whl (140 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m140.6/140.6 KB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: pytz\u003e=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas-\u003edatasets) (2022.7.1)\n","Requirement already satisfied: python-dateutil\u003e=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas-\u003edatasets) (2.8.2)\n","Requirement already satisfied: six\u003e=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil\u003e=2.7.3-\u003epandas-\u003edatasets) (1.15.0)\n","Installing collected packages: tokenizers, sentencepiece, xxhash, urllib3, multiprocess, responses, huggingface-hub, transformers, datasets\n","  Attempting uninstall: urllib3\n","    Found existing installation: urllib3 1.24.3\n","    Uninstalling urllib3-1.24.3:\n","      Successfully uninstalled urllib3-1.24.3\n","Successfully installed datasets-2.9.0 huggingface-hub-0.12.1 multiprocess-0.70.14 responses-0.18.0 sentencepiece-0.1.97 tokenizers-0.13.2 transformers-4.26.1 urllib3-1.26.14 xxhash-3.2.0\n"]}],"source":["!pip install transformers datasets torch sentencepiece"]},{"cell_type":"markdown","metadata":{"id":"dhN0rmb5Oi3d"},"source":["# Załadowanie datasetu"]},{"cell_type":"code","execution_count":2,"metadata":{"executionInfo":{"elapsed":864,"status":"ok","timestamp":1676653735698,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"tnaDkwZ2Pbnn"},"outputs":[],"source":["from datasets import load_dataset"]},{"cell_type":"code","execution_count":3,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":212},"executionInfo":{"elapsed":1523,"status":"ok","timestamp":1676653737219,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"cCiAuRqrOkvV","outputId":"905b8d63-c1d0-4aa2-a87e-d4aab58c0f34"},"outputs":[{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"93b82315618e4f83908ea015b4c75b96","version_major":2,"version_minor":0},"text/plain":["Downloading builder script:   0%|          | 0.00/3.21k [00:00\u003c?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"038fff4f24744940975376531cb7501c","version_major":2,"version_minor":0},"text/plain":["Downloading metadata:   0%|          | 0.00/1.69k [00:00\u003c?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"c310d6f9bed748bfb55df6154ac875be","version_major":2,"version_minor":0},"text/plain":["Downloading readme:   0%|          | 0.00/4.87k [00:00\u003c?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":["Downloading and preparing dataset sms_spam/plain_text to /root/.cache/huggingface/datasets/sms_spam/plain_text/1.0.0/53f051d3b5f62d99d61792c91acefe4f1577ad3e4c216fb0ad39e30b9f20019c...\n"]},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"315e5fdf362f4dc3a004caa623dfb9d0","version_major":2,"version_minor":0},"text/plain":["Downloading data:   0%|          | 0.00/203k [00:00\u003c?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"635c8754690e4427abe06396511acba6","version_major":2,"version_minor":0},"text/plain":["Generating train split:   0%|          | 0/5574 [00:00\u003c?, ? examples/s]"]},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":["Dataset sms_spam downloaded and prepared to /root/.cache/huggingface/datasets/sms_spam/plain_text/1.0.0/53f051d3b5f62d99d61792c91acefe4f1577ad3e4c216fb0ad39e30b9f20019c. Subsequent calls will reuse this data.\n"]},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"cae06d0c71de42c2b2f046d7c864a4d6","version_major":2,"version_minor":0},"text/plain":["  0%|          | 0/1 [00:00\u003c?, ?it/s]"]},"metadata":{},"output_type":"display_data"}],"source":["dataset = load_dataset(\"sms_spam\")"]},{"cell_type":"code","execution_count":4,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":549,"status":"ok","timestamp":1676653737764,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"774MQFXODmgC","outputId":"0ff7f03e-8c6a-4fb1-9449-f9b65496f8ae"},"outputs":[{"name":"stdout","output_type":"stream","text":["Procent wartości z label '0' w zbiorze treningowym: 86.59849300322928%\n","None\n","Procent wartości z label '1' w zbiorze treningowym: 13.40150699677072%\n","None\n"]}],"source":["zero = 0\n","one = 0\n","size_train = 0\n","\n","for row in dataset['train']:\n","  text = row['sms'].replace(\"\\n\", \"\")\n","  new_row = {}\n","  new_row['sms'] = text\n","  if row['label'] == 0:\n","    zero += 1\n","  else:\n","    one += 1\n","  size_train += 1\n","\n","print(print(f\"Procent wartości z label '0' w zbiorze treningowym: {((zero/size_train)*100)}%\"))\n","print(print(f\"Procent wartości z label '1' w zbiorze treningowym: {((one/size_train)*100)}%\"))"]},{"cell_type":"code","execution_count":5,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":4,"status":"ok","timestamp":1676653737764,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"JKFHPko3OnAV","outputId":"28f4f970-7d0d-436e-de0a-fb1552b94ab1"},"outputs":[{"data":{"text/plain":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\\n',\n"," 'label': 0}"]},"execution_count":5,"metadata":{},"output_type":"execute_result"}],"source":["dataset['train'][0]"]},{"cell_type":"markdown","metadata":{"id":"l140vJrgYxPr"},"source":["# Przygotowanie datasetu"]},{"cell_type":"code","execution_count":6,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":356,"status":"ok","timestamp":1676653738118,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"1boUF-YiY3_y","outputId":"9eded657-e64b-4b12-86f2-bb012faf62db"},"outputs":[{"data":{"text/plain":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...',\n"," 'label': 'False'}"]},"execution_count":6,"metadata":{},"output_type":"execute_result"}],"source":["parsed_dataset = []\n","\n","for row in dataset['train']:\n","  text = row['sms'].replace(\"\\n\", \"\")\n","  new_row = {}\n","  new_row['sms'] = text\n","  if row['label'] == 0:\n","    new_row['label'] = \"False\"\n","  else:\n","    new_row['label'] = \"True\"\n","  parsed_dataset.append(new_row)\n","\n","parsed_dataset[0]"]},{"cell_type":"markdown","metadata":{"id":"O-J-jBDxPJcn"},"source":["# Tokenizer FLAN-T5"]},{"cell_type":"code","execution_count":7,"metadata":{"executionInfo":{"elapsed":3339,"status":"ok","timestamp":1676653741455,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"P23AYPX1PZ6g"},"outputs":[],"source":["from transformers import AutoTokenizer"]},{"cell_type":"code","execution_count":8,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":145},"executionInfo":{"elapsed":425,"status":"ok","timestamp":1676653741875,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"q5Jz0E_oPMBr","outputId":"13bfbd68-33c8-4c15-d348-fe4662be91ad"},"outputs":[{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"ee26d66e27dd4b868a2ff8d75db90ae5","version_major":2,"version_minor":0},"text/plain":["Downloading (…)okenizer_config.json:   0%|          | 0.00/2.54k [00:00\u003c?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"fe4f7ee0544f4f92afd12e3068e41b52","version_major":2,"version_minor":0},"text/plain":["Downloading (…)\"spiece.model\";:   0%|          | 0.00/792k [00:00\u003c?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"bc7ecc0725ed48a7b6cefbd1f9244aec","version_major":2,"version_minor":0},"text/plain":["Downloading (…)/main/tokenizer.json:   0%|          | 0.00/2.42M [00:00\u003c?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"9e57149e53064978a7e089d462b50085","version_major":2,"version_minor":0},"text/plain":["Downloading (…)cial_tokens_map.json:   0%|          | 0.00/2.20k [00:00\u003c?, ?B/s]"]},"metadata":{},"output_type":"display_data"}],"source":["tokenizer = AutoTokenizer.from_pretrained('google/flan-t5-large')"]},{"cell_type":"code","execution_count":9,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":12,"status":"ok","timestamp":1676653741875,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"dfxJQpoePsvI","outputId":"b1f93b9d-39d7-4f8f-c611-e666e12d75f2"},"outputs":[{"name":"stdout","output_type":"stream","text":["Original:  Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\n","Tokenized:  ['▁Go', '▁until', '▁jur', 'ong', '▁point', ',', '▁crazy', '.', '.', '▁Available', '▁only', '▁in', '▁bug', 'is', '▁', 'n', '▁great', '▁world', '▁la', '▁', 'e', '▁buffet', '...', '▁Cine', '▁there', '▁got', '▁', 'a', 'more', '▁wa', 't', '...']\n","Token IDs:  [1263, 552, 10081, 2444, 500, 6, 6139, 5, 5, 8144, 163, 16, 8143, 159, 3, 29, 248, 296, 50, 3, 15, 15385, 233, 17270, 132, 530, 3, 9, 3706, 8036, 17, 233]\n"]}],"source":["sms = parsed_dataset[0]['sms']\n","print('Original: ', sms)\n","print('Tokenized: ', tokenizer.tokenize(sms))\n","print('Token IDs: ', tokenizer.convert_tokens_to_ids(tokenizer.tokenize(sms)))"]},{"cell_type":"markdown","metadata":{"id":"UpluhM8cU5Ir"},"source":["# Few shot learning"]},{"cell_type":"code","execution_count":10,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":10,"status":"ok","timestamp":1676653741876,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"7uNUkixPU85O","outputId":"e48646c3-c64a-4aba-dfae-a728e2e18480"},"outputs":[{"name":"stdout","output_type":"stream","text":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...', 'label': 'False'}\n","{'sms': 'Todays Voda numbers ending 7548 are selected to receive a $350 award. If you have a match please call 08712300220 quoting claim code 4041 standard rates app', 'label': 'True'}\n","{'sms': \"LMAO where's your fish memory when I need it?\", 'label': 'False'}\n","{'sms': 'This message is free. Welcome to the new \u0026 improved Sex \u0026 Dogging club! To unsubscribe from this service reply STOP. msgs@150p 18+only', 'label': 'True'}\n"]}],"source":["print(parsed_dataset[0]) #0\n","print(parsed_dataset[123]) #1\n","print(parsed_dataset[2000]) #0\n","print(parsed_dataset[3002]) #1"]},{"cell_type":"code","execution_count":11,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":9,"status":"ok","timestamp":1676653741876,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"lj0issBznZfK","outputId":"159a092e-9702-4afb-e2fc-e284bebfb5ac"},"outputs":[{"name":"stdout","output_type":"stream","text":["Content of the text message: Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\n","Answer if this text message is spam or not: False\n","\n","Content of the text message: Todays Voda numbers ending 7548 are selected to receive a $350 award. If you have a match please call 08712300220 quoting claim code 4041 standard rates app\n","Answer if this text message is spam or not: True\n","\n","Content of the text message: LMAO where's your fish memory when I need it?\n","Answer if this text message is spam or not: False\n","\n","Content of the text message: This message is free. Welcome to the new \u0026 improved Sex \u0026 Dogging club! To unsubscribe from this service reply STOP. msgs@150p 18+only\n","Answer if this text message is spam or not: True\n","\n","Content of the text message: \n"]}],"source":["non_spam_1 = \"Content of the text message: \" + parsed_dataset[0]['sms'] + \"\\nAnswer if this text message is spam or not: False\\n\\n\"\n","spam_1 = \"Content of the text message: \" + parsed_dataset[123]['sms'] + \"\\nAnswer if this text message is spam or not: True\\n\\n\"\n","non_spam_2 = \"Content of the text message: \" + parsed_dataset[2000]['sms'] + \"\\nAnswer if this text message is spam or not: False\\n\\n\"\n","spam_2 = \"Content of the text message: \" + parsed_dataset[3002]['sms'] + \"\\nAnswer if this text message is spam or not: True\\n\\n\"\n","\n","few_shot_prefix = non_spam_1 + spam_1 + non_spam_2 + spam_2 + \"Content of the text message: \"\n","print(few_shot_prefix)"]},{"cell_type":"markdown","metadata":{"id":"okTx_ynMV0rH"},"source":["# Load FLAN-T5 model"]},{"cell_type":"code","execution_count":12,"metadata":{"executionInfo":{"elapsed":8,"status":"ok","timestamp":1676653741877,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"Eu-7Eed8WgN0"},"outputs":[],"source":["from transformers import AutoModelForSeq2SeqLM"]},{"cell_type":"code","execution_count":13,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000},"executionInfo":{"elapsed":35942,"status":"ok","timestamp":1676653777811,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"JKv9O8kfV2zZ","outputId":"41d0d592-5d5f-4fd5-9b91-6fa65b5c9cc2"},"outputs":[{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"a34cf63b12804c8fb770f2d55fa38b0b","version_major":2,"version_minor":0},"text/plain":["Downloading (…)lve/main/config.json:   0%|          | 0.00/662 [00:00\u003c?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"4bb6b6bea7c549508fce2ff2fe74e34b","version_major":2,"version_minor":0},"text/plain":["Downloading (…)\"pytorch_model.bin\";:   0%|          | 0.00/3.13G [00:00\u003c?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"e3bbe227891f4a4281c8fdf98d20c45e","version_major":2,"version_minor":0},"text/plain":["Downloading (…)neration_config.json:   0%|          | 0.00/147 [00:00\u003c?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"text/plain":["T5ForConditionalGeneration(\n","  (shared): Embedding(32128, 1024)\n","  (encoder): T5Stack(\n","    (embed_tokens): Embedding(32128, 1024)\n","    (block): ModuleList(\n","      (0): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","              (relative_attention_bias): Embedding(32, 16)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (1): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (2): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (3): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (4): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (5): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (6): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (7): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (8): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (9): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (10): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (11): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (12): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (13): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (14): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (15): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (16): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (17): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (18): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (19): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (20): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (21): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (22): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (23): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","    )\n","    (final_layer_norm): T5LayerNorm()\n","    (dropout): Dropout(p=0.1, inplace=False)\n","  )\n","  (decoder): T5Stack(\n","    (embed_tokens): Embedding(32128, 1024)\n","    (block): ModuleList(\n","      (0): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","              (relative_attention_bias): Embedding(32, 16)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (1): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (2): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (3): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (4): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (5): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (6): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (7): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (8): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (9): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (10): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (11): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (12): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (13): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (14): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (15): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (16): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (17): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (18): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (19): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (20): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (21): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (22): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (23): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","    )\n","    (final_layer_norm): T5LayerNorm()\n","    (dropout): Dropout(p=0.1, inplace=False)\n","  )\n","  (lm_head): Linear(in_features=1024, out_features=32128, bias=False)\n",")"]},"execution_count":13,"metadata":{},"output_type":"execute_result"}],"source":["model = AutoModelForSeq2SeqLM.from_pretrained('google/flan-t5-large')\n","\n","model.cuda()"]},{"cell_type":"markdown","metadata":{"id":"F_SDAwxoawDy"},"source":["# Helper functions"]},{"cell_type":"code","execution_count":14,"metadata":{"executionInfo":{"elapsed":19,"status":"ok","timestamp":1676653777812,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"rdWMg_KJZEZH"},"outputs":[],"source":["import torch"]},{"cell_type":"code","execution_count":15,"metadata":{"executionInfo":{"elapsed":17,"status":"ok","timestamp":1676653777812,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"FzUi8908ax61"},"outputs":[],"source":["def calculate_accuracy(preds, target):\n","  results_ok = 0.0\n","  results_false = 0.0\n","\n","  for idx, pred in enumerate(preds):\n","    if pred == target[idx]:\n","      results_ok += 1.0\n","    else:\n","      results_false += 1.0\n","\n","  return results_ok / (results_ok + results_false)"]},{"cell_type":"code","execution_count":16,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":18,"status":"ok","timestamp":1676653777813,"user":{"displayName":"Mikołaj Paterka","userId":"04323847504426804830"},"user_tz":-60},"id":"86i7iRmtW-6L","outputId":"53233039-0832-4639-ce27-bbde9d6c9066"},"outputs":[{"name":"stdout","output_type":"stream","text":["There are 1 GPU(s) available.\n","We will use the GPU: Tesla T4\n"]}],"source":["if torch.cuda.is_available():     \n","    device = torch.device(\"cuda\")\n","\n","    print('There are %d GPU(s) available.' % torch.cuda.device_count())\n","    print('We will use the GPU:', torch.cuda.get_device_name(0))\n","\n","else:\n","    print('No GPU available, using the CPU instead.')\n","    device = torch.device(\"cpu\")"]},{"cell_type":"markdown","metadata":{"id":"H_YI3bS3VHQE"},"source":["# Predykcja"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"background_save":true,"base_uri":"https://localhost:8080/"},"id":"vunLGuBGVGmh"},"outputs":[{"name":"stderr","output_type":"stream","text":["Token indices sequence length is longer than the specified maximum sequence length for this model (562 \u003e 512). Running this sequence through the model will result in indexing errors\n"]}],"source":["parsed_dataset = parsed_dataset[1:123] + parsed_dataset[124:2000] + parsed_dataset[2001:3002] + parsed_dataset[3003:]\n","predictions = []\n","expected = []\n","\n","for row in parsed_dataset:\n","  input_text = few_shot_prefix + row['sms'] + \"\\nAnswer if this text message is spam or not: \"\n","  input_ids = tokenizer(input_text, return_tensors=\"pt\").input_ids.to(device)\n","\n","  generated_ids = model.generate(input_ids, do_sample=True, temperature=0.9, max_length=250)\n","  generated_text = tokenizer.decode(generated_ids[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)\n","  \n","  predictions.append(generated_text)\n","  expected.append(row['label'])\n","\n","acc = calculate_accuracy(predictions, expected)\n","print(acc)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"20WbYZmLaDl7"},"outputs":[],"source":["print(\"Sample prediction: {}, expected: {}\".format(predictions[101], expected[101]))"]},{"cell_type":"markdown","metadata":{"id":"dLYc9WXz_B1o"},"source":["# MCC Score"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"hPEPpXXX_DXR"},"outputs":[],"source":["from sklearn.metrics import matthews_corrcoef\n","\n","print('Calculating Matthews Corr. Coef. for each batch...')\n","matthews = matthews_corrcoef(expected, predictions)    \n","print('Total MCC: %.3f' % matthews)"]},{"cell_type":"markdown","metadata":{"id":"GPhCp068_Iwq"},"source":["# Save model"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"avafCMoS_KDF"},"outputs":[],"source":["from google.colab import drive\n","\n","drive.mount('/content/gdrive/', force_remount=True)\n","\n","output_dir = '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_2_FLAN-T5_Model'\n","print(\"Saving model to %s\" % output_dir)\n","\n","model_to_save = model.module if hasattr(model, 'module') else model\n","model_to_save.save_pretrained(output_dir)\n","tokenizer.save_pretrained(output_dir)"]},{"cell_type":"markdown","metadata":{"id":"wHzm2_nDA6i-"},"source":["# Bibliografia\n","- https://huggingface.co/docs/transformers/main/en/model_doc/flan-t5\n","- https://mccormickml.com/2019/07/22/BERT-fine-tuning/#a1-saving--loading-fine-tuned-model\n","- https://huggingface.co/docs/transformers/model_doc/t5#training"]}],"metadata":{"accelerator":"GPU","colab":{"name":"","version":""},"gpuClass":"standard","kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"name":"python"},"widgets":{"application/vnd.jupyter.widget-state+json":{"01fa85b1582d43cb8fc1122a2669f5f9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b04dc96ecf4541d888c53e37038de227","placeholder":"​","style":"IPY_MODEL_5155369b9ec040ce8f7517fdca852dfc","value":" 2.20k/2.20k [00:00\u0026lt;00:00, 142kB/s]"}},"0217aa41498643268cf37d157f0f1ee7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"038fff4f24744940975376531cb7501c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_9acff40e9201484d9efd93b22cbf3cbc","IPY_MODEL_2d4e542615b442d59ecef5f96e229c7a","IPY_MODEL_36f023d53f1c4f25b0ff6fe46891ec0a"],"layout":"IPY_MODEL_b03e0fdba8af4f7086641cb837aa5ac0"}},"056361d016c64e7a8c5433a803eb55e8":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"096221933c7944a18c24af5fd531b267":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_dd7b5391867c4e90843eca09328e2df3","placeholder":"​","style":"IPY_MODEL_ee9736f162744227a561613461339c4d","value":"Downloading data: 100%"}},"0b661568230f453697bf6ab9bfbac49d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"0cd1508476c34d1cba3bdef529e7a49f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"0e5a911a64754176936dfef4643b58e1":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0f5c0a9bb4ca485c8a227bb831ea9500":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"0f5e6960fc754b0eb91ed337f669117c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":"hidden","width":null}},"11d24c25d8c24e4d89a71eeb88dc4b66":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"11f69180b6624687a04d3b05e8d7305f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"","description_tooltip":null,"layout":"IPY_MODEL_236aa0a0b58f4a48ac99873cd944025a","max":5574,"min":0,"orientation":"horizontal","style":"IPY_MODEL_46e1b77919784cebadefb07cba49221b","value":5574}},"14de8872949244df96b1fb4ae9f8b3e3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"162855ef3f784ae68247f0cb8ace63a4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"16c4bd069e42423c8327946c966088b1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_af14a5cdcca04cba8861e8d12d2c8849","placeholder":"​","style":"IPY_MODEL_1cd809fcea1344a5a571bec8b382f73c","value":"Downloading (…)\u0026quot;spiece.model\u0026quot;;: 100%"}},"17cf52e9b6ec44b89e0d1a54b25b4a83":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_aef67cf3439d428ab82f9101683ea1bc","placeholder":"​","style":"IPY_MODEL_162855ef3f784ae68247f0cb8ace63a4","value":" 792k/792k [00:00\u0026lt;00:00, 12.3MB/s]"}},"1901228c1070447c8b0ca1fc1d5f927e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1cd809fcea1344a5a571bec8b382f73c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"225135ae602e4a1a8b3c0a1ade74eca8":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"22c685c95da948adb27cc052b1eba20d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"236aa0a0b58f4a48ac99873cd944025a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"250b9d8b7d7d46ffa7ac553628bff0ba":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_4493d6b76e634615961959d0d4a1f3c9","placeholder":"​","style":"IPY_MODEL_822eb9d2505e4ee4860126696592611e","value":"Downloading (…)\u0026quot;pytorch_model.bin\u0026quot;;: 100%"}},"286b8b517a9f429fbad49dcc133d91f1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_46cfdb5b54c3468a9a2c17dac54ab482","max":2424064,"min":0,"orientation":"horizontal","style":"IPY_MODEL_5aa0f98ba3bf4856ab6ebd1237c8d1a1","value":2424064}},"2bc96a69e0cd4a058c52364ebd2ad185":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2d0ba1f2ea05452c94300e94632a2a21":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2d3683c71e5548d3bc42a4b3fc635789":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_4c96a835345a4e3a8b00ce8a5bb48e2f","max":3132781861,"min":0,"orientation":"horizontal","style":"IPY_MODEL_bf9c12f3b1ec4fc9b8acc748ce6fb6a7","value":3132781861}},"2d4e542615b442d59ecef5f96e229c7a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_3e39eab468d8411ba0966d0769492159","max":1687,"min":0,"orientation":"horizontal","style":"IPY_MODEL_8e552b8e20124239a8442a1674320040","value":1687}},"2d9d143f88ec48738e5f53cf555b5551":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2fd79f08553f4dcf87135bae87892645":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"30a4a25c62be4dec8ce064947eb73b83":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"315e5fdf362f4dc3a004caa623dfb9d0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_096221933c7944a18c24af5fd531b267","IPY_MODEL_4367d3ee219b44c0b48446e3813b0538","IPY_MODEL_e64bdc6fa7dd4707bc859b44219b9179"],"layout":"IPY_MODEL_b5a0765d9001415290422a16131d27f1"}},"33ff65886bc143be8e9c891081f586dd":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"36f023d53f1c4f25b0ff6fe46891ec0a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_acc53b31f1b74aa99e9fc450d0b8b0f1","placeholder":"​","style":"IPY_MODEL_46bd35ed05c74541a89a42ac9d8084e6","value":" 1.69k/1.69k [00:00\u0026lt;00:00, 80.1kB/s]"}},"39134ecbede544ebaaf2c6b63aed4ddc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a5375125da454353b02a0297dcef9bee","placeholder":"​","style":"IPY_MODEL_bc8c98b3819341fd9e4bf104a3460f8d","value":"Downloading (…)cial_tokens_map.json: 100%"}},"3b1edca9c8474aa292c0761683691fa3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7e5baf9772404edfbddf909583885ba9","placeholder":"​","style":"IPY_MODEL_2d0ba1f2ea05452c94300e94632a2a21","value":" 4.87k/4.87k [00:00\u0026lt;00:00, 271kB/s]"}},"3bf72ba781d84fc1bda2a9c5c1c0eeb3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3ccc0ed41f954832a32ae508b7b1cb48":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_70feeae2048f4e408a948b37ab559f30","max":791656,"min":0,"orientation":"horizontal","style":"IPY_MODEL_9977ef6d75814a73b2200317206e84d7","value":791656}},"3e39eab468d8411ba0966d0769492159":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"40b38f21d2c4482392e8b633b26d1e11":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"40ed8dffd64e4e64834b97d7a3d311ee":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4367d3ee219b44c0b48446e3813b0538":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_30a4a25c62be4dec8ce064947eb73b83","max":203415,"min":0,"orientation":"horizontal","style":"IPY_MODEL_a7c84f4d18be4ca1944f80c66bf50edd","value":203415}},"4493d6b76e634615961959d0d4a1f3c9":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"44a25bdd553a4028982d7fdc0ffdac65":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_c88def893c4c469f839de0e36a4d95ea","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1901228c1070447c8b0ca1fc1d5f927e","value":1}},"46bd35ed05c74541a89a42ac9d8084e6":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"46cfdb5b54c3468a9a2c17dac54ab482":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"46e1b77919784cebadefb07cba49221b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"46f0118bfc394bfbaf327bff58a6c5db":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_c21a9fbc4e204f0da4323a8d956aca7d","max":147,"min":0,"orientation":"horizontal","style":"IPY_MODEL_0f5c0a9bb4ca485c8a227bb831ea9500","value":147}},"4bb6b6bea7c549508fce2ff2fe74e34b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_250b9d8b7d7d46ffa7ac553628bff0ba","IPY_MODEL_2d3683c71e5548d3bc42a4b3fc635789","IPY_MODEL_d318c6d470914121a1f18122d03278e1"],"layout":"IPY_MODEL_6ad0879bb99e48389936d9206f8f0f55"}},"4c96a835345a4e3a8b00ce8a5bb48e2f":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4ec65fbdbb044684bb6e4bea3abf2e4b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_924dd3b57fb14513b00b3062ba4e10d9","max":2537,"min":0,"orientation":"horizontal","style":"IPY_MODEL_f37fc0a663b14e39bb8af3472b4e7300","value":2537}},"5003a101b99b4c42af21cef674c272fb":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5155369b9ec040ce8f7517fdca852dfc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"535cd5cf71034a978483592c02581988":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"54a909866f38499791c725a7f0a08d34":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"57ac4fc7c809471cae147e782e9b9127":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5aa0f98ba3bf4856ab6ebd1237c8d1a1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"635c8754690e4427abe06396511acba6":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_746d546cf8e44d3a8d404bc4832f2089","IPY_MODEL_11f69180b6624687a04d3b05e8d7305f","IPY_MODEL_fce9f523028144b28f1e34607c2cf284"],"layout":"IPY_MODEL_0f5e6960fc754b0eb91ed337f669117c"}},"66e5ca18ed294b73ab807cb2d9a3dcc3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"67dd5a3567b4482b811d11655270f504":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f26a05d4fc40452aabb0200c26017cd7","placeholder":"​","style":"IPY_MODEL_e1c42ebbaa1e4402affb87b846d04ce2","value":" 662/662 [00:00\u0026lt;00:00, 28.0kB/s]"}},"6ad0879bb99e48389936d9206f8f0f55":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6c6d417175e841fba41fb4023ecdb676":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_74061316e411484bb7170ef09343218d","placeholder":"​","style":"IPY_MODEL_9dc8c264dfcb4a10a033ead05b2dff82","value":"Downloading readme: 100%"}},"6ca2719906ce4ece92e034605d0069db":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_225135ae602e4a1a8b3c0a1ade74eca8","placeholder":"​","style":"IPY_MODEL_b3d5be3c91744a1fbcbcdb5c687dfebe","value":"100%"}},"6db2327f995146c19e262623a1453705":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"70feeae2048f4e408a948b37ab559f30":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"74061316e411484bb7170ef09343218d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"746d546cf8e44d3a8d404bc4832f2089":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6db2327f995146c19e262623a1453705","placeholder":"​","style":"IPY_MODEL_af15936b6e55483aaa0defb9a822c4f1","value":"Generating train split:  59%"}},"74a88bb92aaa41beabece0d5cb29c8ca":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"75c24f2720084650bafb05e66006c3b7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"783e81b9939c4205b9c033eea726c730":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7bf5bd9be2fe4ddd806aa07eca91268a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7d634b73825b4d9c969560bafed9c693":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_c64f876bff014a94b29b9bb8b9ad7c19","max":3208,"min":0,"orientation":"horizontal","style":"IPY_MODEL_11d24c25d8c24e4d89a71eeb88dc4b66","value":3208}},"7e5baf9772404edfbddf909583885ba9":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"822eb9d2505e4ee4860126696592611e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"84928dab05a24dc299c5f68419e2fb5d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8643946182b94634aab8eaacba273bc3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8b3ddac09d5042039382a7cd817193fc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_2fd79f08553f4dcf87135bae87892645","placeholder":"​","style":"IPY_MODEL_57ac4fc7c809471cae147e782e9b9127","value":" 2.54k/2.54k [00:00\u0026lt;00:00, 101kB/s]"}},"8cecf5ec09ef40f9861c0fc909b69c87":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d69d5236b154469cbdc7f721a72c0626","placeholder":"​","style":"IPY_MODEL_40ed8dffd64e4e64834b97d7a3d311ee","value":"Downloading (…)lve/main/config.json: 100%"}},"8e552b8e20124239a8442a1674320040":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"923f6dbe2d1b4b8b853632990303db99":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"924dd3b57fb14513b00b3062ba4e10d9":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"93b82315618e4f83908ea015b4c75b96":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_de88e7aee35840dbad51b2fa30b0b9b1","IPY_MODEL_7d634b73825b4d9c969560bafed9c693","IPY_MODEL_f27e6ad6f1af4859bf202fc81cdcd7ab"],"layout":"IPY_MODEL_923f6dbe2d1b4b8b853632990303db99"}},"95db20de12f149a8bddc258be32f4fc1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"9977ef6d75814a73b2200317206e84d7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"9acff40e9201484d9efd93b22cbf3cbc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5003a101b99b4c42af21cef674c272fb","placeholder":"​","style":"IPY_MODEL_b574912d6be34230b13fd059c1324c62","value":"Downloading metadata: 100%"}},"9dc8c264dfcb4a10a033ead05b2dff82":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9e57149e53064978a7e089d462b50085":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_39134ecbede544ebaaf2c6b63aed4ddc","IPY_MODEL_a1613d578e7a4c5289eca005763b7d47","IPY_MODEL_01fa85b1582d43cb8fc1122a2669f5f9"],"layout":"IPY_MODEL_f2d3bd7fe97d42289b6de8343ebcf4d9"}},"9f1e4e705e4348cab6557e52bf178957":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_75c24f2720084650bafb05e66006c3b7","placeholder":"​","style":"IPY_MODEL_22c685c95da948adb27cc052b1eba20d","value":" 1/1 [00:00\u0026lt;00:00, 44.30it/s]"}},"a1613d578e7a4c5289eca005763b7d47":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a956b2e7dc3542b796db3913fd754923","max":2201,"min":0,"orientation":"horizontal","style":"IPY_MODEL_0cd1508476c34d1cba3bdef529e7a49f","value":2201}},"a31a0d3a579542bd9936fc72562d91a8":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a34cf63b12804c8fb770f2d55fa38b0b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8cecf5ec09ef40f9861c0fc909b69c87","IPY_MODEL_d73412a5e22e42beb1f219e0ab154eeb","IPY_MODEL_67dd5a3567b4482b811d11655270f504"],"layout":"IPY_MODEL_3bf72ba781d84fc1bda2a9c5c1c0eeb3"}},"a3d76642349140df8b7d370cddb91278":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_f027f0a305a94a78bb37269d2176ca5b","max":4872,"min":0,"orientation":"horizontal","style":"IPY_MODEL_95db20de12f149a8bddc258be32f4fc1","value":4872}},"a5375125da454353b02a0297dcef9bee":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a7c84f4d18be4ca1944f80c66bf50edd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"a886387b419947ba8ed64f405ac4411b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a956b2e7dc3542b796db3913fd754923":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"abda8f31317f43118a0c4498322a86fd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e403a2746b3e4ed9a59abe38820f3b17","placeholder":"​","style":"IPY_MODEL_66e5ca18ed294b73ab807cb2d9a3dcc3","value":"Downloading (…)/main/tokenizer.json: 100%"}},"acc53b31f1b74aa99e9fc450d0b8b0f1":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"aef67cf3439d428ab82f9101683ea1bc":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"af14a5cdcca04cba8861e8d12d2c8849":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"af15936b6e55483aaa0defb9a822c4f1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b03e0fdba8af4f7086641cb837aa5ac0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b04dc96ecf4541d888c53e37038de227":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b1d49dbe41574009b31a2d9023e6991b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b2c0c37150174cc29c096bd9637cfac1":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b3d5be3c91744a1fbcbcdb5c687dfebe":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b42c311aa2ad4086bea865504765a9ed":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_74a88bb92aaa41beabece0d5cb29c8ca","placeholder":"​","style":"IPY_MODEL_feaaa3fb7c3743b9ba7107c7b25e6548","value":"Downloading (…)okenizer_config.json: 100%"}},"b574912d6be34230b13fd059c1324c62":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b5a0765d9001415290422a16131d27f1":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b8c05d3fb439473c9745b02a688b032a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"bc7ecc0725ed48a7b6cefbd1f9244aec":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_abda8f31317f43118a0c4498322a86fd","IPY_MODEL_286b8b517a9f429fbad49dcc133d91f1","IPY_MODEL_f6ea563561bc4d58802489b3a15f564e"],"layout":"IPY_MODEL_b2c0c37150174cc29c096bd9637cfac1"}},"bc8c98b3819341fd9e4bf104a3460f8d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"bf104cc787124caebc93820d3fceb8f4":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bf9c12f3b1ec4fc9b8acc748ce6fb6a7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"c21a9fbc4e204f0da4323a8d956aca7d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c310d6f9bed748bfb55df6154ac875be":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_6c6d417175e841fba41fb4023ecdb676","IPY_MODEL_a3d76642349140df8b7d370cddb91278","IPY_MODEL_3b1edca9c8474aa292c0761683691fa3"],"layout":"IPY_MODEL_535cd5cf71034a978483592c02581988"}},"c64f876bff014a94b29b9bb8b9ad7c19":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c88def893c4c469f839de0e36a4d95ea":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cae06d0c71de42c2b2f046d7c864a4d6":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_6ca2719906ce4ece92e034605d0069db","IPY_MODEL_44a25bdd553a4028982d7fdc0ffdac65","IPY_MODEL_9f1e4e705e4348cab6557e52bf178957"],"layout":"IPY_MODEL_8643946182b94634aab8eaacba273bc3"}},"d318c6d470914121a1f18122d03278e1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_fd5b22f075624e23a59683e9f40faa67","placeholder":"​","style":"IPY_MODEL_b8c05d3fb439473c9745b02a688b032a","value":" 3.13G/3.13G [00:15\u0026lt;00:00, 191MB/s]"}},"d5667e9594ed4e91afaf171e4b716928":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d57f118544c14d6fafde81abd13b0f91":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_056361d016c64e7a8c5433a803eb55e8","placeholder":"​","style":"IPY_MODEL_a886387b419947ba8ed64f405ac4411b","value":"Downloading (…)neration_config.json: 100%"}},"d69d5236b154469cbdc7f721a72c0626":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d73412a5e22e42beb1f219e0ab154eeb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_54a909866f38499791c725a7f0a08d34","max":662,"min":0,"orientation":"horizontal","style":"IPY_MODEL_0217aa41498643268cf37d157f0f1ee7","value":662}},"dd7b5391867c4e90843eca09328e2df3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"de88e7aee35840dbad51b2fa30b0b9b1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_783e81b9939c4205b9c033eea726c730","placeholder":"​","style":"IPY_MODEL_b1d49dbe41574009b31a2d9023e6991b","value":"Downloading builder script: 100%"}},"e1c42ebbaa1e4402affb87b846d04ce2":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"e3bbe227891f4a4281c8fdf98d20c45e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_d57f118544c14d6fafde81abd13b0f91","IPY_MODEL_46f0118bfc394bfbaf327bff58a6c5db","IPY_MODEL_e498ede224374360919085ca3dca7e96"],"layout":"IPY_MODEL_84928dab05a24dc299c5f68419e2fb5d"}},"e403a2746b3e4ed9a59abe38820f3b17":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e498ede224374360919085ca3dca7e96":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_14de8872949244df96b1fb4ae9f8b3e3","placeholder":"​","style":"IPY_MODEL_0b661568230f453697bf6ab9bfbac49d","value":" 147/147 [00:00\u0026lt;00:00, 6.00kB/s]"}},"e64bdc6fa7dd4707bc859b44219b9179":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0e5a911a64754176936dfef4643b58e1","placeholder":"​","style":"IPY_MODEL_40b38f21d2c4482392e8b633b26d1e11","value":" 203k/203k [00:00\u0026lt;00:00, 327kB/s]"}},"ee26d66e27dd4b868a2ff8d75db90ae5":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b42c311aa2ad4086bea865504765a9ed","IPY_MODEL_4ec65fbdbb044684bb6e4bea3abf2e4b","IPY_MODEL_8b3ddac09d5042039382a7cd817193fc"],"layout":"IPY_MODEL_33ff65886bc143be8e9c891081f586dd"}},"ee9736f162744227a561613461339c4d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f027f0a305a94a78bb37269d2176ca5b":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f26a05d4fc40452aabb0200c26017cd7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f27e6ad6f1af4859bf202fc81cdcd7ab":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f4485f7300bf4321b9caf43083788049","placeholder":"​","style":"IPY_MODEL_d5667e9594ed4e91afaf171e4b716928","value":" 3.21k/3.21k [00:00\u0026lt;00:00, 189kB/s]"}},"f2d3bd7fe97d42289b6de8343ebcf4d9":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f37fc0a663b14e39bb8af3472b4e7300":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"f4485f7300bf4321b9caf43083788049":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f6ea563561bc4d58802489b3a15f564e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_bf104cc787124caebc93820d3fceb8f4","placeholder":"​","style":"IPY_MODEL_2d9d143f88ec48738e5f53cf555b5551","value":" 2.42M/2.42M [00:00\u0026lt;00:00, 61.7MB/s]"}},"fce9f523028144b28f1e34607c2cf284":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7bf5bd9be2fe4ddd806aa07eca91268a","placeholder":"​","style":"IPY_MODEL_a31a0d3a579542bd9936fc72562d91a8","value":" 3284/5574 [00:00\u0026lt;00:00, 23962.99 examples/s]"}},"fd5b22f075624e23a59683e9f40faa67":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fe4f7ee0544f4f92afd12e3068e41b52":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_16c4bd069e42423c8327946c966088b1","IPY_MODEL_3ccc0ed41f954832a32ae508b7b1cb48","IPY_MODEL_17cf52e9b6ec44b89e0d1a54b25b4a83"],"layout":"IPY_MODEL_2bc96a69e0cd4a058c52364ebd2ad185"}},"feaaa3fb7c3743b9ba7107c7b25e6548":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}}},"nbformat":4,"nbformat_minor":0}
\ No newline at end of file
diff --git a/projekt/Ver_4_FLAN-T5_sms_spam.ipynb b/projekt/Ver_4_FLAN-T5_sms_spam.ipynb
new file mode 100644
index 0000000..6838ac5
--- /dev/null
+++ b/projekt/Ver_4_FLAN-T5_sms_spam.ipynb
@@ -0,0 +1 @@
+{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"},"gpuClass":"standard","widgets":{"application/vnd.jupyter.widget-state+json":{"68bd37fea6b9499daec358ce41d9d678":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_c2be6591c7ff4d4daf26ccd12f1844af","IPY_MODEL_adf02588e5184537a93169222740dd3d","IPY_MODEL_622e16eeafb74e939e0c69d492aa604e"],"layout":"IPY_MODEL_18d424bf759a4996984075b098834129"}},"c2be6591c7ff4d4daf26ccd12f1844af":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_48546da0c5504023b1209c01b25d4e4c","placeholder":"​","style":"IPY_MODEL_c95964f713a8408593c599c189dc4ad7","value":"Downloading builder script: 100%"}},"adf02588e5184537a93169222740dd3d":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_61c14a3d13524b4aa5ed7942390fb1da","max":3208,"min":0,"orientation":"horizontal","style":"IPY_MODEL_c1104de7b8db49789ec58adfc7bfe215","value":3208}},"622e16eeafb74e939e0c69d492aa604e":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e3eb39ab40684052b3558938d2955d1b","placeholder":"​","style":"IPY_MODEL_5ad9640c999948cf930de2c024c7c81c","value":" 3.21k/3.21k [00:00&lt;00:00, 157kB/s]"}},"18d424bf759a4996984075b098834129":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"48546da0c5504023b1209c01b25d4e4c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c95964f713a8408593c599c189dc4ad7":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"61c14a3d13524b4aa5ed7942390fb1da":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c1104de7b8db49789ec58adfc7bfe215":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"e3eb39ab40684052b3558938d2955d1b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5ad9640c999948cf930de2c024c7c81c":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"361b829c01d94f028fd511602401b630":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b0500d4eb22e470382d692e1dc91d6b5","IPY_MODEL_58912ae770724c56a55212c37502572b","IPY_MODEL_d5ad570f7f4f4cb49b258001d4ac1b58"],"layout":"IPY_MODEL_d0ec3fa6ab724b368df4ab862c5b7b1b"}},"b0500d4eb22e470382d692e1dc91d6b5":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_246e37be5003475a8e682e7797fa7db0","placeholder":"​","style":"IPY_MODEL_ddce8b1ecabe492684b70a3fd8fad029","value":"Downloading metadata: 100%"}},"58912ae770724c56a55212c37502572b":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a254ee7123ec411cb0076b6a15397281","max":1687,"min":0,"orientation":"horizontal","style":"IPY_MODEL_d239e3573e904b51b834cfa5d6e9e3af","value":1687}},"d5ad570f7f4f4cb49b258001d4ac1b58":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e58081a3ec5141ae88bf49a08e3771b5","placeholder":"​","style":"IPY_MODEL_ece5328c3eab49ea8148efca475f5eb8","value":" 1.69k/1.69k [00:00&lt;00:00, 30.0kB/s]"}},"d0ec3fa6ab724b368df4ab862c5b7b1b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"246e37be5003475a8e682e7797fa7db0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ddce8b1ecabe492684b70a3fd8fad029":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a254ee7123ec411cb0076b6a15397281":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d239e3573e904b51b834cfa5d6e9e3af":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"e58081a3ec5141ae88bf49a08e3771b5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ece5328c3eab49ea8148efca475f5eb8":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"12e031e63c4b4753b19c2f262cf6550c":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_664490b6e48c4e09a37957ea2a32705c","IPY_MODEL_ed9152228e864e02b956dc345ced59ad","IPY_MODEL_130c35b0aef541409fdd1e7b6d33004a"],"layout":"IPY_MODEL_ef4757d97869447f8192346626730277"}},"664490b6e48c4e09a37957ea2a32705c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_932e3c5942ac450ab7245d370b410a39","placeholder":"​","style":"IPY_MODEL_09ed8501e44441b0af5fa6d549abb776","value":"Downloading readme: 100%"}},"ed9152228e864e02b956dc345ced59ad":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_4e4f6641562a4741afdefebfb04c3ca3","max":4872,"min":0,"orientation":"horizontal","style":"IPY_MODEL_a48363c76a424855b178ef5ecc6be3c0","value":4872}},"130c35b0aef541409fdd1e7b6d33004a":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_904f59209b524551bd629944d870bd44","placeholder":"​","style":"IPY_MODEL_721fa15e35844743880b72ab03243309","value":" 4.87k/4.87k [00:00&lt;00:00, 234kB/s]"}},"ef4757d97869447f8192346626730277":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"932e3c5942ac450ab7245d370b410a39":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"09ed8501e44441b0af5fa6d549abb776":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4e4f6641562a4741afdefebfb04c3ca3":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a48363c76a424855b178ef5ecc6be3c0":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"904f59209b524551bd629944d870bd44":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"721fa15e35844743880b72ab03243309":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6557e65a8adc45efab4ef47a50653bf0":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_0adbedaa248e4eabb6110fde0e32b75e","IPY_MODEL_42156b19be1848e7b6ea19874821fbb9","IPY_MODEL_9911649dc95b46c2a9c04be1e0fa7861"],"layout":"IPY_MODEL_0423daad31da47979f84c546d698480c"}},"0adbedaa248e4eabb6110fde0e32b75e":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7e452481918f485f9901108b8dc9db77","placeholder":"​","style":"IPY_MODEL_9049707729a14ff1862939f7aec876ab","value":"Downloading data: 100%"}},"42156b19be1848e7b6ea19874821fbb9":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_ddc0eb506b544e85b999f11566162679","max":203415,"min":0,"orientation":"horizontal","style":"IPY_MODEL_a0cf1620ed6b4ea39d57690f7a2c273c","value":203415}},"9911649dc95b46c2a9c04be1e0fa7861":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_bf7d7c639ca942228ae14179798b5821","placeholder":"​","style":"IPY_MODEL_840d8a832c8f4a25960ffed3705ce99b","value":" 203k/203k [00:00&lt;00:00, 749kB/s]"}},"0423daad31da47979f84c546d698480c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7e452481918f485f9901108b8dc9db77":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9049707729a14ff1862939f7aec876ab":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ddc0eb506b544e85b999f11566162679":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a0cf1620ed6b4ea39d57690f7a2c273c":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"bf7d7c639ca942228ae14179798b5821":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"840d8a832c8f4a25960ffed3705ce99b":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4ac215662d6f40a28a7998911fb617f0":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_75d0debcd9104328b5ce905d0cf95d05","IPY_MODEL_d47951f821724274a88cd5f157e9224d","IPY_MODEL_82982684fd9341ac92223f30cbea8412"],"layout":"IPY_MODEL_c9df05e8f8f94302b1a61129a0b79ab6"}},"75d0debcd9104328b5ce905d0cf95d05":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_df3bc60cf37d422bbe8e55a9093499e5","placeholder":"​","style":"IPY_MODEL_c1e6db93577a478d9192fd4a9c03a439","value":"Generating train split: 100%"}},"d47951f821724274a88cd5f157e9224d":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"","description_tooltip":null,"layout":"IPY_MODEL_9e61eda06ddf43a8997f4ca69f58dbdb","max":5574,"min":0,"orientation":"horizontal","style":"IPY_MODEL_11e8310afe6e4d6288d1e7e285a880bb","value":5574}},"82982684fd9341ac92223f30cbea8412":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e42e05102ce144758c58874f21c9e2f2","placeholder":"​","style":"IPY_MODEL_c9b324922b424052b3d9cefadc921c27","value":" 5574/5574 [00:00&lt;00:00, 15020.97 examples/s]"}},"c9df05e8f8f94302b1a61129a0b79ab6":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":"hidden","width":null}},"df3bc60cf37d422bbe8e55a9093499e5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c1e6db93577a478d9192fd4a9c03a439":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9e61eda06ddf43a8997f4ca69f58dbdb":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"11e8310afe6e4d6288d1e7e285a880bb":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"e42e05102ce144758c58874f21c9e2f2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c9b324922b424052b3d9cefadc921c27":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6190bdcfdbad409698f1d0f525d4325f":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_81478adafe094101aa5444721a77013a","IPY_MODEL_423f9593f9894680bbf4dcc4dd2ef724","IPY_MODEL_25d37d0fbc3a4917b4421b3b67f9c4e5"],"layout":"IPY_MODEL_be15f6e1f9d74ac5be7457e4c6edccd0"}},"81478adafe094101aa5444721a77013a":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_19f31506f4d140beb5c7ac6076f83b47","placeholder":"​","style":"IPY_MODEL_8858842842fe4aa3be0c11e80aa65d3a","value":"100%"}},"423f9593f9894680bbf4dcc4dd2ef724":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_78d9b1024b214602b0a8c6d15d35a89d","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_678aa3d231d44c0c9ffb6c38e9e095f6","value":1}},"25d37d0fbc3a4917b4421b3b67f9c4e5":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e789a3898f8c41c2bdf8ea5748b18104","placeholder":"​","style":"IPY_MODEL_eea178b6106448bca7ea2b9f2eae6527","value":" 1/1 [00:00&lt;00:00, 31.02it/s]"}},"be15f6e1f9d74ac5be7457e4c6edccd0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"19f31506f4d140beb5c7ac6076f83b47":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8858842842fe4aa3be0c11e80aa65d3a":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"78d9b1024b214602b0a8c6d15d35a89d":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"678aa3d231d44c0c9ffb6c38e9e095f6":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"e789a3898f8c41c2bdf8ea5748b18104":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"eea178b6106448bca7ea2b9f2eae6527":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6a27116f01b648ef942141677975405d":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_6640b97f0817419c98a633e83eb854f6","IPY_MODEL_17f50238e442461b873192ed1a7e5673","IPY_MODEL_0bc47b507ae64766958ec4062359d962"],"layout":"IPY_MODEL_c4b010d66ba4491083f26a1cd0fd0b09"}},"6640b97f0817419c98a633e83eb854f6":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9f6d9358f00445518e118e084bd8c192","placeholder":"​","style":"IPY_MODEL_8b97c4b26c4145c58a8efc1cfd31042c","value":"Downloading (…)okenizer_config.json: 100%"}},"17f50238e442461b873192ed1a7e5673":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_565f2dc27a8c43a6b85b87efa558bdf5","max":2539,"min":0,"orientation":"horizontal","style":"IPY_MODEL_0e8794c1d221420d9a306e7d35a3d63e","value":2539}},"0bc47b507ae64766958ec4062359d962":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_8d04dd43e8024043b63e98b7caa3c97b","placeholder":"​","style":"IPY_MODEL_099201033c6e461088795edc27470c09","value":" 2.54k/2.54k [00:00&lt;00:00, 68.8kB/s]"}},"c4b010d66ba4491083f26a1cd0fd0b09":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9f6d9358f00445518e118e084bd8c192":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8b97c4b26c4145c58a8efc1cfd31042c":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"565f2dc27a8c43a6b85b87efa558bdf5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0e8794c1d221420d9a306e7d35a3d63e":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"8d04dd43e8024043b63e98b7caa3c97b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"099201033c6e461088795edc27470c09":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d195065e792b496fa0674d96d80bc5e7":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_ef3744a453a74cbc907467a121cabdc7","IPY_MODEL_59c8487d5d1441c2ad888c96b62b2fc6","IPY_MODEL_c88f9cfc8c8846499bac0faace319921"],"layout":"IPY_MODEL_438c39c7c97e4172a8c4e4c29e0dde86"}},"ef3744a453a74cbc907467a121cabdc7":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e4244f9bd21a4f8caa038e1922cdac85","placeholder":"​","style":"IPY_MODEL_53d371fdcd4c47cdbb60b68e14693c18","value":"Downloading (…)&quot;spiece.model&quot;;: 100%"}},"59c8487d5d1441c2ad888c96b62b2fc6":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_24225d17f2c24048aba70043851fc616","max":791656,"min":0,"orientation":"horizontal","style":"IPY_MODEL_7f13470e9f874a859edd3ad81dc2eef3","value":791656}},"c88f9cfc8c8846499bac0faace319921":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_65503a9084df4525a199b732b8b56478","placeholder":"​","style":"IPY_MODEL_20a51f65184b41878c4938609b48ca6c","value":" 792k/792k [00:00&lt;00:00, 6.51MB/s]"}},"438c39c7c97e4172a8c4e4c29e0dde86":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e4244f9bd21a4f8caa038e1922cdac85":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"53d371fdcd4c47cdbb60b68e14693c18":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"24225d17f2c24048aba70043851fc616":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7f13470e9f874a859edd3ad81dc2eef3":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"65503a9084df4525a199b732b8b56478":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"20a51f65184b41878c4938609b48ca6c":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2ea815fd8b3f4d0883166e93ad3c5289":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_1361a74b2aef49ea98deafe7027bd568","IPY_MODEL_d532b5fe357a401da221671ff12c8904","IPY_MODEL_a8e5ad44d09c4e2189719bef46e7a39f"],"layout":"IPY_MODEL_0e0e1632da714b19b8baef34c1210473"}},"1361a74b2aef49ea98deafe7027bd568":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_aba6113b613d458abd74686aeae13657","placeholder":"​","style":"IPY_MODEL_729afdc7fb37406ca334d8fb8a77e58a","value":"Downloading (…)/main/tokenizer.json: 100%"}},"d532b5fe357a401da221671ff12c8904":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_3b65129b567b44019e0b37c8401b8ed2","max":2424064,"min":0,"orientation":"horizontal","style":"IPY_MODEL_ed353f256ab6492891bf10d32f051e45","value":2424064}},"a8e5ad44d09c4e2189719bef46e7a39f":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_bbaf656fa94f413b959797d47353a049","placeholder":"​","style":"IPY_MODEL_b659b1d8cc544a608200d5ef5076818e","value":" 2.42M/2.42M [00:00&lt;00:00, 17.7MB/s]"}},"0e0e1632da714b19b8baef34c1210473":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"aba6113b613d458abd74686aeae13657":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"729afdc7fb37406ca334d8fb8a77e58a":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3b65129b567b44019e0b37c8401b8ed2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ed353f256ab6492891bf10d32f051e45":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"bbaf656fa94f413b959797d47353a049":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b659b1d8cc544a608200d5ef5076818e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"497cde8c9a81401f92afa18ba9a6ce40":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_45ad6503850542a8a2f7a3b0479e88ca","IPY_MODEL_d276f99074864c0cbcd8daa17e5f6d2f","IPY_MODEL_04cebfae2d984f07a97351116037e3cf"],"layout":"IPY_MODEL_c94595332c0c4f18afde731efbaba866"}},"45ad6503850542a8a2f7a3b0479e88ca":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_587bdb3634a24c638696bcd9fe21368c","placeholder":"​","style":"IPY_MODEL_dc450f9305da4afcbb4e6a1bf77c3590","value":"Downloading (…)cial_tokens_map.json: 100%"}},"d276f99074864c0cbcd8daa17e5f6d2f":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_884ace4175f94127b1674352d17e67f2","max":2201,"min":0,"orientation":"horizontal","style":"IPY_MODEL_3901be176ed144b986f20a7a7a72f8e2","value":2201}},"04cebfae2d984f07a97351116037e3cf":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7259afa2646a414ebb90f486242e3661","placeholder":"​","style":"IPY_MODEL_a09e4534caef4b43862c275a59617c6b","value":" 2.20k/2.20k [00:00&lt;00:00, 115kB/s]"}},"c94595332c0c4f18afde731efbaba866":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"587bdb3634a24c638696bcd9fe21368c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"dc450f9305da4afcbb4e6a1bf77c3590":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"884ace4175f94127b1674352d17e67f2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3901be176ed144b986f20a7a7a72f8e2":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"7259afa2646a414ebb90f486242e3661":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a09e4534caef4b43862c275a59617c6b":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8e0c9ce977054a879a1929e1c030a6a3":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_bde5c3406f0e46d8ab30a8b9f1355d5c","IPY_MODEL_91505f004c3742e68e9a70155fedcf2e","IPY_MODEL_0a7e5af5e2b64c488432b9563401c299"],"layout":"IPY_MODEL_4c8fd9d5d7364812900f4e35490948fd"}},"bde5c3406f0e46d8ab30a8b9f1355d5c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_64d4b67445c34a6182c2b1ec07339b22","placeholder":"​","style":"IPY_MODEL_c4f9e0afdb334188abaee11b383a0142","value":"Downloading (…)lve/main/config.json: 100%"}},"91505f004c3742e68e9a70155fedcf2e":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_8ce37c30b02646148b9694a14e21fbce","max":662,"min":0,"orientation":"horizontal","style":"IPY_MODEL_6a3fff3870b34941b4713c984c3c681c","value":662}},"0a7e5af5e2b64c488432b9563401c299":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5ba546d284b04dee96c132985a8bd38c","placeholder":"​","style":"IPY_MODEL_534b173a13c544e48af4051d748b89b8","value":" 662/662 [00:00&lt;00:00, 20.6kB/s]"}},"4c8fd9d5d7364812900f4e35490948fd":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"64d4b67445c34a6182c2b1ec07339b22":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c4f9e0afdb334188abaee11b383a0142":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8ce37c30b02646148b9694a14e21fbce":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6a3fff3870b34941b4713c984c3c681c":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"5ba546d284b04dee96c132985a8bd38c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"534b173a13c544e48af4051d748b89b8":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2c7e377fed4241fbab8730fd850f5905":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_1c7b9fccd53043c0a9d84ae3c6caabd4","IPY_MODEL_e0423e68cdf64c8cb0e2643c20c8f3cd","IPY_MODEL_440dbb34f66a4ea8b4138da6dcb7d269"],"layout":"IPY_MODEL_d55e16b317a348fdb8f677cc5cab99cb"}},"1c7b9fccd53043c0a9d84ae3c6caabd4":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f17117c9c86141e89c394b598f1c391a","placeholder":"​","style":"IPY_MODEL_7635e183b25f42499d438febea66e8ea","value":"Downloading (…)&quot;pytorch_model.bin&quot;;: 100%"}},"e0423e68cdf64c8cb0e2643c20c8f3cd":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_bcc42f2b6d3847b5b9127503f09c805d","max":3132781861,"min":0,"orientation":"horizontal","style":"IPY_MODEL_95b7176944d74bed8d65239f0dd572cc","value":3132781861}},"440dbb34f66a4ea8b4138da6dcb7d269":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a680a0a6142f4b86bd4b926320ded47c","placeholder":"​","style":"IPY_MODEL_3a80de03299742baa8a05a6f6dc11e65","value":" 3.13G/3.13G [00:28&lt;00:00, 104MB/s]"}},"d55e16b317a348fdb8f677cc5cab99cb":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f17117c9c86141e89c394b598f1c391a":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7635e183b25f42499d438febea66e8ea":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"bcc42f2b6d3847b5b9127503f09c805d":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"95b7176944d74bed8d65239f0dd572cc":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"a680a0a6142f4b86bd4b926320ded47c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3a80de03299742baa8a05a6f6dc11e65":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c707fedbf7564e6195c9339dd850a29a":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_9de390b272414e62a761112ab5f6f2c5","IPY_MODEL_97c6386f6734441cbf30b8cd92e618fe","IPY_MODEL_70dfc5bbac0145e097ff98485435ae37"],"layout":"IPY_MODEL_d18f22d2d3274b57a622ffc8aed5b745"}},"9de390b272414e62a761112ab5f6f2c5":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_452dc83d4775438dbd6f5d70b0a9c281","placeholder":"​","style":"IPY_MODEL_24ce51b9d54f4203bd402280ad319bbd","value":"Downloading (…)neration_config.json: 100%"}},"97c6386f6734441cbf30b8cd92e618fe":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_582ac43518ca4ea7ae2a34e0cd705471","max":147,"min":0,"orientation":"horizontal","style":"IPY_MODEL_48eed9b0ef6a45b9a32e22b468a14d60","value":147}},"70dfc5bbac0145e097ff98485435ae37":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f8372f868b054d3798eaf1e1f4c40838","placeholder":"​","style":"IPY_MODEL_105d6b2a260848849b6fed4593d4a8d8","value":" 147/147 [00:00&lt;00:00, 4.65kB/s]"}},"d18f22d2d3274b57a622ffc8aed5b745":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"452dc83d4775438dbd6f5d70b0a9c281":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"24ce51b9d54f4203bd402280ad319bbd":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"582ac43518ca4ea7ae2a34e0cd705471":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"48eed9b0ef6a45b9a32e22b468a14d60":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"f8372f868b054d3798eaf1e1f4c40838":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"105d6b2a260848849b6fed4593d4a8d8":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","source":["# Instalacja pakietów"],"metadata":{"id":"ZXsOR6oJOJbd"}},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"8l0hzptKNiZS","executionInfo":{"status":"ok","timestamp":1676654821493,"user_tz":-60,"elapsed":23173,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"outputId":"37414816-ea90-4a00-fe06-20fab2ab1b8f"},"outputs":[{"output_type":"stream","name":"stdout","text":["Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n","Collecting transformers\n","  Downloading transformers-4.26.1-py3-none-any.whl (6.3 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.3/6.3 MB\u001b[0m \u001b[31m34.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting datasets\n","  Downloading datasets-2.9.0-py3-none-any.whl (462 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m462.8/462.8 KB\u001b[0m \u001b[31m27.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: torch in /usr/local/lib/python3.8/dist-packages (1.13.1+cu116)\n","Collecting sentencepiece\n","  Downloading sentencepiece-0.1.97-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m52.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.8/dist-packages (from transformers) (4.64.1)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers) (3.9.0)\n","Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (2022.6.2)\n","Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (6.0)\n","Collecting huggingface-hub<1.0,>=0.11.0\n","  Downloading huggingface_hub-0.12.1-py3-none-any.whl (190 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.3/190.3 KB\u001b[0m \u001b[31m15.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (1.21.6)\n","Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from transformers) (2.25.1)\n","Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (23.0)\n","Collecting tokenizers!=0.11.3,<0.14,>=0.11.1\n","  Downloading tokenizers-0.13.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.6 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.6/7.6 MB\u001b[0m \u001b[31m55.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting responses<0.19\n","  Downloading responses-0.18.0-py3-none-any.whl (38 kB)\n","Requirement already satisfied: dill<0.3.7 in /usr/local/lib/python3.8/dist-packages (from datasets) (0.3.6)\n","Requirement already satisfied: pandas in /usr/local/lib/python3.8/dist-packages (from datasets) (1.3.5)\n","Requirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.8/dist-packages (from datasets) (2023.1.0)\n","Collecting multiprocess\n","  Downloading multiprocess-0.70.14-py38-none-any.whl (132 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m132.0/132.0 KB\u001b[0m \u001b[31m8.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: pyarrow>=6.0.0 in /usr/local/lib/python3.8/dist-packages (from datasets) (9.0.0)\n","Requirement already satisfied: aiohttp in /usr/local/lib/python3.8/dist-packages (from datasets) (3.8.3)\n","Collecting xxhash\n","  Downloading xxhash-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (213 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m213.0/213.0 KB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: typing-extensions in /usr/local/lib/python3.8/dist-packages (from torch) (4.4.0)\n","Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (2.1.1)\n","Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (22.2.0)\n","Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.3)\n","Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.1)\n","Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.8.2)\n","Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (4.0.2)\n","Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (6.0.4)\n","Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (4.0.0)\n","Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (1.24.3)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2022.12.7)\n","Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2.10)\n","Collecting urllib3<1.27,>=1.21.1\n","  Downloading urllib3-1.26.14-py2.py3-none-any.whl (140 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m140.6/140.6 KB\u001b[0m \u001b[31m10.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas->datasets) (2.8.2)\n","Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas->datasets) (2022.7.1)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil>=2.7.3->pandas->datasets) (1.15.0)\n","Installing collected packages: tokenizers, sentencepiece, xxhash, urllib3, multiprocess, responses, huggingface-hub, transformers, datasets\n","  Attempting uninstall: urllib3\n","    Found existing installation: urllib3 1.24.3\n","    Uninstalling urllib3-1.24.3:\n","      Successfully uninstalled urllib3-1.24.3\n","Successfully installed datasets-2.9.0 huggingface-hub-0.12.1 multiprocess-0.70.14 responses-0.18.0 sentencepiece-0.1.97 tokenizers-0.13.2 transformers-4.26.1 urllib3-1.26.14 xxhash-3.2.0\n"]}],"source":["!pip install transformers datasets torch sentencepiece"]},{"cell_type":"markdown","source":["# Załadowanie datasetu"],"metadata":{"id":"dhN0rmb5Oi3d"}},{"cell_type":"code","source":["from datasets import load_dataset"],"metadata":{"id":"tnaDkwZ2Pbnn","executionInfo":{"status":"ok","timestamp":1676654822469,"user_tz":-60,"elapsed":980,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}}},"execution_count":2,"outputs":[]},{"cell_type":"code","source":["dataset = load_dataset(\"sms_spam\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":264,"referenced_widgets":["68bd37fea6b9499daec358ce41d9d678","c2be6591c7ff4d4daf26ccd12f1844af","adf02588e5184537a93169222740dd3d","622e16eeafb74e939e0c69d492aa604e","18d424bf759a4996984075b098834129","48546da0c5504023b1209c01b25d4e4c","c95964f713a8408593c599c189dc4ad7","61c14a3d13524b4aa5ed7942390fb1da","c1104de7b8db49789ec58adfc7bfe215","e3eb39ab40684052b3558938d2955d1b","5ad9640c999948cf930de2c024c7c81c","361b829c01d94f028fd511602401b630","b0500d4eb22e470382d692e1dc91d6b5","58912ae770724c56a55212c37502572b","d5ad570f7f4f4cb49b258001d4ac1b58","d0ec3fa6ab724b368df4ab862c5b7b1b","246e37be5003475a8e682e7797fa7db0","ddce8b1ecabe492684b70a3fd8fad029","a254ee7123ec411cb0076b6a15397281","d239e3573e904b51b834cfa5d6e9e3af","e58081a3ec5141ae88bf49a08e3771b5","ece5328c3eab49ea8148efca475f5eb8","12e031e63c4b4753b19c2f262cf6550c","664490b6e48c4e09a37957ea2a32705c","ed9152228e864e02b956dc345ced59ad","130c35b0aef541409fdd1e7b6d33004a","ef4757d97869447f8192346626730277","932e3c5942ac450ab7245d370b410a39","09ed8501e44441b0af5fa6d549abb776","4e4f6641562a4741afdefebfb04c3ca3","a48363c76a424855b178ef5ecc6be3c0","904f59209b524551bd629944d870bd44","721fa15e35844743880b72ab03243309","6557e65a8adc45efab4ef47a50653bf0","0adbedaa248e4eabb6110fde0e32b75e","42156b19be1848e7b6ea19874821fbb9","9911649dc95b46c2a9c04be1e0fa7861","0423daad31da47979f84c546d698480c","7e452481918f485f9901108b8dc9db77","9049707729a14ff1862939f7aec876ab","ddc0eb506b544e85b999f11566162679","a0cf1620ed6b4ea39d57690f7a2c273c","bf7d7c639ca942228ae14179798b5821","840d8a832c8f4a25960ffed3705ce99b","4ac215662d6f40a28a7998911fb617f0","75d0debcd9104328b5ce905d0cf95d05","d47951f821724274a88cd5f157e9224d","82982684fd9341ac92223f30cbea8412","c9df05e8f8f94302b1a61129a0b79ab6","df3bc60cf37d422bbe8e55a9093499e5","c1e6db93577a478d9192fd4a9c03a439","9e61eda06ddf43a8997f4ca69f58dbdb","11e8310afe6e4d6288d1e7e285a880bb","e42e05102ce144758c58874f21c9e2f2","c9b324922b424052b3d9cefadc921c27","6190bdcfdbad409698f1d0f525d4325f","81478adafe094101aa5444721a77013a","423f9593f9894680bbf4dcc4dd2ef724","25d37d0fbc3a4917b4421b3b67f9c4e5","be15f6e1f9d74ac5be7457e4c6edccd0","19f31506f4d140beb5c7ac6076f83b47","8858842842fe4aa3be0c11e80aa65d3a","78d9b1024b214602b0a8c6d15d35a89d","678aa3d231d44c0c9ffb6c38e9e095f6","e789a3898f8c41c2bdf8ea5748b18104","eea178b6106448bca7ea2b9f2eae6527"]},"id":"cCiAuRqrOkvV","executionInfo":{"status":"ok","timestamp":1676654825207,"user_tz":-60,"elapsed":2740,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"outputId":"d33d75f6-a30c-4fb1-c510-e9cba7cfe403"},"execution_count":3,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading builder script:   0%|          | 0.00/3.21k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"68bd37fea6b9499daec358ce41d9d678"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading metadata:   0%|          | 0.00/1.69k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"361b829c01d94f028fd511602401b630"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading readme:   0%|          | 0.00/4.87k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"12e031e63c4b4753b19c2f262cf6550c"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Downloading and preparing dataset sms_spam/plain_text to /root/.cache/huggingface/datasets/sms_spam/plain_text/1.0.0/53f051d3b5f62d99d61792c91acefe4f1577ad3e4c216fb0ad39e30b9f20019c...\n"]},{"output_type":"display_data","data":{"text/plain":["Downloading data:   0%|          | 0.00/203k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6557e65a8adc45efab4ef47a50653bf0"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Generating train split:   0%|          | 0/5574 [00:00<?, ? examples/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"4ac215662d6f40a28a7998911fb617f0"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Dataset sms_spam downloaded and prepared to /root/.cache/huggingface/datasets/sms_spam/plain_text/1.0.0/53f051d3b5f62d99d61792c91acefe4f1577ad3e4c216fb0ad39e30b9f20019c. Subsequent calls will reuse this data.\n"]},{"output_type":"display_data","data":{"text/plain":["  0%|          | 0/1 [00:00<?, ?it/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6190bdcfdbad409698f1d0f525d4325f"}},"metadata":{}}]},{"cell_type":"code","source":["dataset['train'][0]"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"JKFHPko3OnAV","executionInfo":{"status":"ok","timestamp":1676654825208,"user_tz":-60,"elapsed":10,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"outputId":"c26256d4-313b-4993-d98a-27bcde4f71a5"},"execution_count":4,"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\\n',\n"," 'label': 0}"]},"metadata":{},"execution_count":4}]},{"cell_type":"markdown","source":["# Przygotowanie datasetu"],"metadata":{"id":"l140vJrgYxPr"}},{"cell_type":"code","source":["parsed_dataset = []\n","\n","for row in dataset['train']:\n","  text = row['sms'].replace(\"\\n\", \"\")\n","  new_row = {}\n","  new_row['sms'] = text\n","  if row['label'] == 0:\n","    new_row['label'] = \"False\"\n","  else:\n","    new_row['label'] = \"True\"\n","  parsed_dataset.append(new_row)\n","\n","parsed_dataset[0]"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"1boUF-YiY3_y","executionInfo":{"status":"ok","timestamp":1676654825208,"user_tz":-60,"elapsed":8,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"outputId":"fca5e196-7cb5-4c25-8fbb-e8becc020a88"},"execution_count":5,"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...',\n"," 'label': 'False'}"]},"metadata":{},"execution_count":5}]},{"cell_type":"markdown","source":["# Tokenizer FLAN-T5"],"metadata":{"id":"O-J-jBDxPJcn"}},{"cell_type":"code","source":["from transformers import AutoTokenizer"],"metadata":{"id":"P23AYPX1PZ6g","executionInfo":{"status":"ok","timestamp":1676654827189,"user_tz":-60,"elapsed":1987,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}}},"execution_count":6,"outputs":[]},{"cell_type":"code","source":["tokenizer = AutoTokenizer.from_pretrained('google/flan-t5-large')"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":145,"referenced_widgets":["6a27116f01b648ef942141677975405d","6640b97f0817419c98a633e83eb854f6","17f50238e442461b873192ed1a7e5673","0bc47b507ae64766958ec4062359d962","c4b010d66ba4491083f26a1cd0fd0b09","9f6d9358f00445518e118e084bd8c192","8b97c4b26c4145c58a8efc1cfd31042c","565f2dc27a8c43a6b85b87efa558bdf5","0e8794c1d221420d9a306e7d35a3d63e","8d04dd43e8024043b63e98b7caa3c97b","099201033c6e461088795edc27470c09","d195065e792b496fa0674d96d80bc5e7","ef3744a453a74cbc907467a121cabdc7","59c8487d5d1441c2ad888c96b62b2fc6","c88f9cfc8c8846499bac0faace319921","438c39c7c97e4172a8c4e4c29e0dde86","e4244f9bd21a4f8caa038e1922cdac85","53d371fdcd4c47cdbb60b68e14693c18","24225d17f2c24048aba70043851fc616","7f13470e9f874a859edd3ad81dc2eef3","65503a9084df4525a199b732b8b56478","20a51f65184b41878c4938609b48ca6c","2ea815fd8b3f4d0883166e93ad3c5289","1361a74b2aef49ea98deafe7027bd568","d532b5fe357a401da221671ff12c8904","a8e5ad44d09c4e2189719bef46e7a39f","0e0e1632da714b19b8baef34c1210473","aba6113b613d458abd74686aeae13657","729afdc7fb37406ca334d8fb8a77e58a","3b65129b567b44019e0b37c8401b8ed2","ed353f256ab6492891bf10d32f051e45","bbaf656fa94f413b959797d47353a049","b659b1d8cc544a608200d5ef5076818e","497cde8c9a81401f92afa18ba9a6ce40","45ad6503850542a8a2f7a3b0479e88ca","d276f99074864c0cbcd8daa17e5f6d2f","04cebfae2d984f07a97351116037e3cf","c94595332c0c4f18afde731efbaba866","587bdb3634a24c638696bcd9fe21368c","dc450f9305da4afcbb4e6a1bf77c3590","884ace4175f94127b1674352d17e67f2","3901be176ed144b986f20a7a7a72f8e2","7259afa2646a414ebb90f486242e3661","a09e4534caef4b43862c275a59617c6b"]},"id":"q5Jz0E_oPMBr","executionInfo":{"status":"ok","timestamp":1676654828626,"user_tz":-60,"elapsed":1440,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"outputId":"5f1635b6-2904-4994-9161-4a3ecafc6896"},"execution_count":7,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading (…)okenizer_config.json:   0%|          | 0.00/2.54k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6a27116f01b648ef942141677975405d"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)\"spiece.model\";:   0%|          | 0.00/792k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"d195065e792b496fa0674d96d80bc5e7"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)/main/tokenizer.json:   0%|          | 0.00/2.42M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"2ea815fd8b3f4d0883166e93ad3c5289"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)cial_tokens_map.json:   0%|          | 0.00/2.20k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"497cde8c9a81401f92afa18ba9a6ce40"}},"metadata":{}}]},{"cell_type":"code","source":["sms = parsed_dataset[0]['sms']\n","print('Original: ', sms)\n","print('Tokenized: ', tokenizer.tokenize(sms))\n","print('Token IDs: ', tokenizer.convert_tokens_to_ids(tokenizer.tokenize(sms)))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"dfxJQpoePsvI","executionInfo":{"status":"ok","timestamp":1676654828626,"user_tz":-60,"elapsed":10,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"outputId":"fccd3b61-53c9-49ed-f480-f5894e926059"},"execution_count":8,"outputs":[{"output_type":"stream","name":"stdout","text":["Original:  Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\n","Tokenized:  ['▁Go', '▁until', '▁jur', 'ong', '▁point', ',', '▁crazy', '.', '.', '▁Available', '▁only', '▁in', '▁bug', 'is', '▁', 'n', '▁great', '▁world', '▁la', '▁', 'e', '▁buffet', '...', '▁Cine', '▁there', '▁got', '▁', 'a', 'more', '▁wa', 't', '...']\n","Token IDs:  [1263, 552, 10081, 2444, 500, 6, 6139, 5, 5, 8144, 163, 16, 8143, 159, 3, 29, 248, 296, 50, 3, 15, 15385, 233, 17270, 132, 530, 3, 9, 3706, 8036, 17, 233]\n"]}]},{"cell_type":"markdown","source":["# Few shot learning"],"metadata":{"id":"UpluhM8cU5Ir"}},{"cell_type":"code","source":["print(parsed_dataset[0]) #0\n","print(parsed_dataset[123]) #1\n","print(parsed_dataset[2000]) #0\n","print(parsed_dataset[3002]) #1"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"7uNUkixPU85O","executionInfo":{"status":"ok","timestamp":1676654828627,"user_tz":-60,"elapsed":9,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"outputId":"56579b1f-a2f7-4de5-83c7-d181b2cee335"},"execution_count":9,"outputs":[{"output_type":"stream","name":"stdout","text":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...', 'label': 'False'}\n","{'sms': 'Todays Voda numbers ending 7548 are selected to receive a $350 award. If you have a match please call 08712300220 quoting claim code 4041 standard rates app', 'label': 'True'}\n","{'sms': \"LMAO where's your fish memory when I need it?\", 'label': 'False'}\n","{'sms': 'This message is free. Welcome to the new & improved Sex & Dogging club! To unsubscribe from this service reply STOP. msgs@150p 18+only', 'label': 'True'}\n"]}]},{"cell_type":"code","source":["non_spam_1 = \"The text message contains the following content: \" + parsed_dataset[0]['sms'] + \"\\nIs it true that the content of a text message is spam: False\\n\\n\"\n","spam_1 = \"The text message contains the following content: \" + parsed_dataset[123]['sms'] + \"\\nIs it true that the content of a text message is spam: True\\n\\n\"\n","non_spam_2 = \"The text message contains the following content: \" + parsed_dataset[2000]['sms'] + \"\\nIs it true that the content of a text message is spam: False\\n\\n\"\n","spam_2 = \"The text message contains the following content: \" + parsed_dataset[3002]['sms'] + \"\\nIs it true that the content of a text message is spam: True\\n\\n\"\n","\n","few_shot_prefix = non_spam_1 + spam_1 + non_spam_2 + spam_2 + \"The text message contains the following content: \"\n","print(few_shot_prefix)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"lj0issBznZfK","executionInfo":{"status":"ok","timestamp":1676654828627,"user_tz":-60,"elapsed":7,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"outputId":"4e55029a-d838-45fa-c08c-98b957ceb642"},"execution_count":10,"outputs":[{"output_type":"stream","name":"stdout","text":["The text message contains the following content: Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\n","Is it true that the content of a text message is spam: False\n","\n","The text message contains the following content: Todays Voda numbers ending 7548 are selected to receive a $350 award. If you have a match please call 08712300220 quoting claim code 4041 standard rates app\n","Is it true that the content of a text message is spam: True\n","\n","The text message contains the following content: LMAO where's your fish memory when I need it?\n","Is it true that the content of a text message is spam: False\n","\n","The text message contains the following content: This message is free. Welcome to the new & improved Sex & Dogging club! To unsubscribe from this service reply STOP. msgs@150p 18+only\n","Is it true that the content of a text message is spam: True\n","\n","The text message contains the following content: \n"]}]},{"cell_type":"markdown","source":["# Load FLAN-T5 model"],"metadata":{"id":"okTx_ynMV0rH"}},{"cell_type":"code","source":["from transformers import AutoModelForSeq2SeqLM"],"metadata":{"id":"Eu-7Eed8WgN0","executionInfo":{"status":"ok","timestamp":1676654828628,"user_tz":-60,"elapsed":6,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}}},"execution_count":11,"outputs":[]},{"cell_type":"code","source":["model = AutoModelForSeq2SeqLM.from_pretrained('google/flan-t5-large')\n","\n","model.cuda()"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000,"referenced_widgets":["8e0c9ce977054a879a1929e1c030a6a3","bde5c3406f0e46d8ab30a8b9f1355d5c","91505f004c3742e68e9a70155fedcf2e","0a7e5af5e2b64c488432b9563401c299","4c8fd9d5d7364812900f4e35490948fd","64d4b67445c34a6182c2b1ec07339b22","c4f9e0afdb334188abaee11b383a0142","8ce37c30b02646148b9694a14e21fbce","6a3fff3870b34941b4713c984c3c681c","5ba546d284b04dee96c132985a8bd38c","534b173a13c544e48af4051d748b89b8","2c7e377fed4241fbab8730fd850f5905","1c7b9fccd53043c0a9d84ae3c6caabd4","e0423e68cdf64c8cb0e2643c20c8f3cd","440dbb34f66a4ea8b4138da6dcb7d269","d55e16b317a348fdb8f677cc5cab99cb","f17117c9c86141e89c394b598f1c391a","7635e183b25f42499d438febea66e8ea","bcc42f2b6d3847b5b9127503f09c805d","95b7176944d74bed8d65239f0dd572cc","a680a0a6142f4b86bd4b926320ded47c","3a80de03299742baa8a05a6f6dc11e65","c707fedbf7564e6195c9339dd850a29a","9de390b272414e62a761112ab5f6f2c5","97c6386f6734441cbf30b8cd92e618fe","70dfc5bbac0145e097ff98485435ae37","d18f22d2d3274b57a622ffc8aed5b745","452dc83d4775438dbd6f5d70b0a9c281","24ce51b9d54f4203bd402280ad319bbd","582ac43518ca4ea7ae2a34e0cd705471","48eed9b0ef6a45b9a32e22b468a14d60","f8372f868b054d3798eaf1e1f4c40838","105d6b2a260848849b6fed4593d4a8d8"]},"id":"JKv9O8kfV2zZ","executionInfo":{"status":"ok","timestamp":1676654878825,"user_tz":-60,"elapsed":50202,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"outputId":"802742eb-cd94-4500-f299-13dc8cd41887"},"execution_count":12,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading (…)lve/main/config.json:   0%|          | 0.00/662 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"8e0c9ce977054a879a1929e1c030a6a3"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)\"pytorch_model.bin\";:   0%|          | 0.00/3.13G [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"2c7e377fed4241fbab8730fd850f5905"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)neration_config.json:   0%|          | 0.00/147 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"c707fedbf7564e6195c9339dd850a29a"}},"metadata":{}},{"output_type":"execute_result","data":{"text/plain":["T5ForConditionalGeneration(\n","  (shared): Embedding(32128, 1024)\n","  (encoder): T5Stack(\n","    (embed_tokens): Embedding(32128, 1024)\n","    (block): ModuleList(\n","      (0): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","              (relative_attention_bias): Embedding(32, 16)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (1): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (2): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (3): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (4): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (5): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (6): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (7): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (8): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (9): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (10): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (11): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (12): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (13): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (14): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (15): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (16): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (17): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (18): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (19): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (20): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (21): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (22): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (23): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","    )\n","    (final_layer_norm): T5LayerNorm()\n","    (dropout): Dropout(p=0.1, inplace=False)\n","  )\n","  (decoder): T5Stack(\n","    (embed_tokens): Embedding(32128, 1024)\n","    (block): ModuleList(\n","      (0): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","              (relative_attention_bias): Embedding(32, 16)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (1): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (2): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (3): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (4): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (5): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (6): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (7): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (8): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (9): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (10): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (11): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (12): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (13): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (14): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (15): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (16): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (17): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (18): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (19): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (20): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (21): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (22): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (23): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","    )\n","    (final_layer_norm): T5LayerNorm()\n","    (dropout): Dropout(p=0.1, inplace=False)\n","  )\n","  (lm_head): Linear(in_features=1024, out_features=32128, bias=False)\n",")"]},"metadata":{},"execution_count":12}]},{"cell_type":"markdown","source":["# Helper functions"],"metadata":{"id":"F_SDAwxoawDy"}},{"cell_type":"code","source":["import torch"],"metadata":{"id":"rdWMg_KJZEZH","executionInfo":{"status":"ok","timestamp":1676654878826,"user_tz":-60,"elapsed":16,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}}},"execution_count":13,"outputs":[]},{"cell_type":"code","source":["def calculate_accuracy(preds, target):\n","  results_ok = 0.0\n","  results_false = 0.0\n","\n","  for idx, pred in enumerate(preds):\n","    if pred == target[idx]:\n","      results_ok += 1.0\n","    else:\n","      results_false += 1.0\n","\n","  return results_ok / (results_ok + results_false)"],"metadata":{"id":"FzUi8908ax61","executionInfo":{"status":"ok","timestamp":1676654878826,"user_tz":-60,"elapsed":15,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}}},"execution_count":14,"outputs":[]},{"cell_type":"code","source":["if torch.cuda.is_available():     \n","    device = torch.device(\"cuda\")\n","\n","    print('There are %d GPU(s) available.' % torch.cuda.device_count())\n","    print('We will use the GPU:', torch.cuda.get_device_name(0))\n","\n","else:\n","    print('No GPU available, using the CPU instead.')\n","    device = torch.device(\"cpu\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"86i7iRmtW-6L","executionInfo":{"status":"ok","timestamp":1676654878826,"user_tz":-60,"elapsed":15,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"outputId":"66e7c490-7cc0-4665-b342-284b55329d16"},"execution_count":15,"outputs":[{"output_type":"stream","name":"stdout","text":["There are 1 GPU(s) available.\n","We will use the GPU: Tesla T4\n"]}]},{"cell_type":"markdown","source":["# Predykcja"],"metadata":{"id":"H_YI3bS3VHQE"}},{"cell_type":"code","source":["parsed_dataset = parsed_dataset[1:123] + parsed_dataset[124:2000] + parsed_dataset[2001:3002] + parsed_dataset[3003:]\n","predictions = []\n","expected = []\n","\n","for row in parsed_dataset:\n","  input_text = few_shot_prefix + row['sms'] + \"\\nIs it true that the content of a text message is spam: \"\n","  input_ids = tokenizer(input_text, return_tensors=\"pt\").input_ids.to(device)\n","\n","  generated_ids = model.generate(input_ids, do_sample=True, temperature=0.9, max_length=250)\n","  generated_text = tokenizer.decode(generated_ids[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)\n","  \n","  predictions.append(generated_text)\n","  expected.append(row['label'])\n","\n","acc = calculate_accuracy(predictions, expected)\n","print(acc)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"vunLGuBGVGmh","executionInfo":{"status":"ok","timestamp":1676656236490,"user_tz":-60,"elapsed":905220,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"outputId":"9cf4e39e-ebbc-4f36-9a07-17a392ddb8d2"},"execution_count":17,"outputs":[{"output_type":"stream","name":"stdout","text":["0.1433704635285663\n"]}]},{"cell_type":"code","source":["print(\"Sample prediction: {}, expected: {}\".format(predictions[101], expected[101]))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"20WbYZmLaDl7","executionInfo":{"status":"ok","timestamp":1676656236491,"user_tz":-60,"elapsed":8,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"outputId":"b1109d93-fbaa-42d4-9b36-c302e9e9f1ff"},"execution_count":18,"outputs":[{"output_type":"stream","name":"stdout","text":["Sample prediction: False, expected: False\n"]}]},{"cell_type":"markdown","source":["# MCC Score"],"metadata":{"id":"dLYc9WXz_B1o"}},{"cell_type":"code","source":["from sklearn.metrics import matthews_corrcoef\n","\n","print('Calculating Matthews Corr. Coef. for each batch...')\n","matthews = matthews_corrcoef(expected, predictions)    \n","print('Total MCC: %.3f' % matthews)"],"metadata":{"id":"hPEPpXXX_DXR","executionInfo":{"status":"ok","timestamp":1676656237402,"user_tz":-60,"elapsed":915,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"d3862278-19af-4c01-85ef-51cfc63bbf9f"},"execution_count":19,"outputs":[{"output_type":"stream","name":"stdout","text":["Calculating Matthews Corr. Coef. for each batch...\n","Total MCC: 0.055\n"]}]},{"cell_type":"markdown","source":["# Save model"],"metadata":{"id":"GPhCp068_Iwq"}},{"cell_type":"code","source":["from google.colab import drive\n","\n","drive.mount('/content/gdrive/', force_remount=True)\n","\n","output_dir = '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_4_FLAN-T5_Model'\n","print(\"Saving model to %s\" % output_dir)\n","\n","model_to_save = model.module if hasattr(model, 'module') else model\n","model_to_save.save_pretrained(output_dir)\n","tokenizer.save_pretrained(output_dir)"],"metadata":{"id":"avafCMoS_KDF","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1676656370608,"user_tz":-60,"elapsed":133210,"user":{"displayName":"Mike2 Studia2","userId":"10506861792614965645"}},"outputId":"11a9bbd2-c5aa-4147-bc0d-633cda86b955"},"execution_count":20,"outputs":[{"output_type":"stream","name":"stdout","text":["Mounted at /content/gdrive/\n","Saving model to /content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_4_FLAN-T5_Model\n"]},{"output_type":"execute_result","data":{"text/plain":["('/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_4_FLAN-T5_Model/tokenizer_config.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_4_FLAN-T5_Model/special_tokens_map.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_4_FLAN-T5_Model/spiece.model',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_4_FLAN-T5_Model/added_tokens.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_4_FLAN-T5_Model/tokenizer.json')"]},"metadata":{},"execution_count":20}]},{"cell_type":"markdown","source":["# Bibliografia\n","- https://huggingface.co/docs/transformers/main/en/model_doc/flan-t5\n","- https://mccormickml.com/2019/07/22/BERT-fine-tuning/#a1-saving--loading-fine-tuned-model\n","- https://huggingface.co/docs/transformers/model_doc/t5#training"],"metadata":{"id":"wHzm2_nDA6i-"}}]}
\ No newline at end of file
diff --git a/projekt/Ver_5_FLAN-T5_sms_spam.ipynb b/projekt/Ver_5_FLAN-T5_sms_spam.ipynb
new file mode 100644
index 0000000..cccbce2
--- /dev/null
+++ b/projekt/Ver_5_FLAN-T5_sms_spam.ipynb
@@ -0,0 +1 @@
+{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"},"gpuClass":"standard","widgets":{"application/vnd.jupyter.widget-state+json":{"b6eded70136441759cc5ac10b00acb8f":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_c5f3c896b6534c178ae822c32a8e00b3","IPY_MODEL_fe35cc2de5b840e49d9d9c5fec869656","IPY_MODEL_532adf3b2f7a4b75a095e9455351553c"],"layout":"IPY_MODEL_646379eb229040dcb09aa0f03d6f3f26"}},"c5f3c896b6534c178ae822c32a8e00b3":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5ee0acafd6944686b9ef3dedabed6aef","placeholder":"​","style":"IPY_MODEL_e3f4a5f8f4864b0d8357aa29d2b8802f","value":"Downloading builder script: 100%"}},"fe35cc2de5b840e49d9d9c5fec869656":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_9152a884650a486f9275278678061eec","max":3208,"min":0,"orientation":"horizontal","style":"IPY_MODEL_9e9b5088b6054ba2a48c6b7872a2c383","value":3208}},"532adf3b2f7a4b75a095e9455351553c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f222becd734644c4ae631d432e1b3156","placeholder":"​","style":"IPY_MODEL_510afccfa74748c4bd69a10dc53e3b70","value":" 3.21k/3.21k [00:00&lt;00:00, 154kB/s]"}},"646379eb229040dcb09aa0f03d6f3f26":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5ee0acafd6944686b9ef3dedabed6aef":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e3f4a5f8f4864b0d8357aa29d2b8802f":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9152a884650a486f9275278678061eec":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9e9b5088b6054ba2a48c6b7872a2c383":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"f222becd734644c4ae631d432e1b3156":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"510afccfa74748c4bd69a10dc53e3b70":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"e7db60e680f04318a706fcb1fe7dbcb7":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_101d73ffbf8c47af9f62103648edc548","IPY_MODEL_5a5500aa9fd84381be7676f570665d7a","IPY_MODEL_76c1dca21a3c4c3c91769c48a4428ba3"],"layout":"IPY_MODEL_cee9b746e101471cb5808e2415091f38"}},"101d73ffbf8c47af9f62103648edc548":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_292d1d346d9246f99b87709fac58b0f0","placeholder":"​","style":"IPY_MODEL_94ddbe64d035427d943606085a697f23","value":"Downloading metadata: 100%"}},"5a5500aa9fd84381be7676f570665d7a":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_8f9eb5f919144f74856d354642cff880","max":1687,"min":0,"orientation":"horizontal","style":"IPY_MODEL_176ebdce409b49d59423292931d8a5df","value":1687}},"76c1dca21a3c4c3c91769c48a4428ba3":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a35c33cb3774427f8e739a7b1c948f4e","placeholder":"​","style":"IPY_MODEL_ea9bb434237643ab8bb4e4d6f884ba67","value":" 1.69k/1.69k [00:00&lt;00:00, 81.1kB/s]"}},"cee9b746e101471cb5808e2415091f38":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"292d1d346d9246f99b87709fac58b0f0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"94ddbe64d035427d943606085a697f23":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8f9eb5f919144f74856d354642cff880":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"176ebdce409b49d59423292931d8a5df":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"a35c33cb3774427f8e739a7b1c948f4e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ea9bb434237643ab8bb4e4d6f884ba67":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b1810e9efa944f08ba50be015456287f":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_76abc09e7e794eef9ffc3081b2ec2da6","IPY_MODEL_2fbff5b358734b579a1b1d471934150d","IPY_MODEL_4a0b0fc9717b4fc6bf5ad887785dec90"],"layout":"IPY_MODEL_af367d23351544f187286a069b469e89"}},"76abc09e7e794eef9ffc3081b2ec2da6":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_dd56ef64cae84c3f8dcb52b86bfeeede","placeholder":"​","style":"IPY_MODEL_107cf57f7ba34a7b82a99035098cd60a","value":"Downloading readme: 100%"}},"2fbff5b358734b579a1b1d471934150d":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_d3f4f7df0b8f41b7a11bf9ca7a2c02d1","max":4872,"min":0,"orientation":"horizontal","style":"IPY_MODEL_f0f50dea946240a79adce466c2810c5a","value":4872}},"4a0b0fc9717b4fc6bf5ad887785dec90":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0d06ef4561694b2e8a5c495f55f01320","placeholder":"​","style":"IPY_MODEL_d8f9577b8ef8418597e3e327ad29e119","value":" 4.87k/4.87k [00:00&lt;00:00, 226kB/s]"}},"af367d23351544f187286a069b469e89":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"dd56ef64cae84c3f8dcb52b86bfeeede":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"107cf57f7ba34a7b82a99035098cd60a":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d3f4f7df0b8f41b7a11bf9ca7a2c02d1":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f0f50dea946240a79adce466c2810c5a":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"0d06ef4561694b2e8a5c495f55f01320":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d8f9577b8ef8418597e3e327ad29e119":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"97a4213ad47f47aaae8dce4947c541d5":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_4ae43dc6645b4e7abfd5018b40b6e1cf","IPY_MODEL_1f2721589fd549269dd8c04fb00ac0b1","IPY_MODEL_cfe42f1c148c4507b8bed4eade47d2cd"],"layout":"IPY_MODEL_a91105129f0a4d0daf1f92beee7b696b"}},"4ae43dc6645b4e7abfd5018b40b6e1cf":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ec41724b9add4285bc5d04715505675b","placeholder":"​","style":"IPY_MODEL_a1ef5bde067c4250b5d5b74ce8b2945d","value":"Downloading data: 100%"}},"1f2721589fd549269dd8c04fb00ac0b1":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_79810b4bd66442aba12328e39a00e438","max":203415,"min":0,"orientation":"horizontal","style":"IPY_MODEL_5f0c3fbda713440d80ec4d0654b5892d","value":203415}},"cfe42f1c148c4507b8bed4eade47d2cd":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_78e6dfa135804ed785e0a809356c5112","placeholder":"​","style":"IPY_MODEL_5fe984999000415b8770abe7c6d65bfc","value":" 203k/203k [00:00&lt;00:00, 879kB/s]"}},"a91105129f0a4d0daf1f92beee7b696b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ec41724b9add4285bc5d04715505675b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a1ef5bde067c4250b5d5b74ce8b2945d":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"79810b4bd66442aba12328e39a00e438":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5f0c3fbda713440d80ec4d0654b5892d":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"78e6dfa135804ed785e0a809356c5112":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5fe984999000415b8770abe7c6d65bfc":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f53b16108c93439d9943b35c28d57988":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8a180e9d3cef4d58a5b05509edb39ec4","IPY_MODEL_d97533a9d431471c898cbccb93202900","IPY_MODEL_235bdfd6aee042a587647925981a1fd2"],"layout":"IPY_MODEL_e081d46be72b4d638cf3cc7be14ac651"}},"8a180e9d3cef4d58a5b05509edb39ec4":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0154f8d5e1ef4584b2def2e456179951","placeholder":"​","style":"IPY_MODEL_d2d1b1c19e8e4b18af10bffe9c3a60f5","value":"Generating train split:  76%"}},"d97533a9d431471c898cbccb93202900":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"","description_tooltip":null,"layout":"IPY_MODEL_40b5e64a2d6743cbbb6b3dca004836b9","max":5574,"min":0,"orientation":"horizontal","style":"IPY_MODEL_e9a816019ddc4c34a94059381da2b7b0","value":5574}},"235bdfd6aee042a587647925981a1fd2":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e0a987f764274098ace0ce3900e37c5e","placeholder":"​","style":"IPY_MODEL_e5b7397c80a44e66863276ed6e443dd9","value":" 4221/5574 [00:00&lt;00:00, 18163.85 examples/s]"}},"e081d46be72b4d638cf3cc7be14ac651":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":"hidden","width":null}},"0154f8d5e1ef4584b2def2e456179951":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d2d1b1c19e8e4b18af10bffe9c3a60f5":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"40b5e64a2d6743cbbb6b3dca004836b9":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e9a816019ddc4c34a94059381da2b7b0":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"e0a987f764274098ace0ce3900e37c5e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e5b7397c80a44e66863276ed6e443dd9":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"594b168dc8f24064afca0a0353195c35":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b641fc8e31514c2e9c9453ba3669575c","IPY_MODEL_c575ee9e9a4846669a70be243ec8a228","IPY_MODEL_48ac88309d404e1cb83e660a8b344ac6"],"layout":"IPY_MODEL_0e48acebc7b04eb2a1b55a0d852ef7b3"}},"b641fc8e31514c2e9c9453ba3669575c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_edcf26a3835b4db6b3fbfd160b61aa2e","placeholder":"​","style":"IPY_MODEL_3eaee86b67704d3da78285c7881cff01","value":"100%"}},"c575ee9e9a4846669a70be243ec8a228":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_167ad19c7da2466db0528865578fda87","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_5c384622a46e4252a123378d29bc7640","value":1}},"48ac88309d404e1cb83e660a8b344ac6":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a60f375b3eee415388d742c619966776","placeholder":"​","style":"IPY_MODEL_39f30b2919dc4ac0b6e3c8d54999a628","value":" 1/1 [00:00&lt;00:00, 22.49it/s]"}},"0e48acebc7b04eb2a1b55a0d852ef7b3":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"edcf26a3835b4db6b3fbfd160b61aa2e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3eaee86b67704d3da78285c7881cff01":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"167ad19c7da2466db0528865578fda87":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5c384622a46e4252a123378d29bc7640":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"a60f375b3eee415388d742c619966776":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"39f30b2919dc4ac0b6e3c8d54999a628":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"964a243415be46aba8b943310ab54d4f":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_d9ccbf0d26014aa584f3e2d0d0e64a68","IPY_MODEL_5634196dc0bd4424843e7d721d28362a","IPY_MODEL_c72e22a6ab40424286a185ac758a225e"],"layout":"IPY_MODEL_c9ca5157a2e44cbaa02cb0314ac8e8bf"}},"d9ccbf0d26014aa584f3e2d0d0e64a68":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_048af3db70184a679800034200d93952","placeholder":"​","style":"IPY_MODEL_52380821e49d448887f932add9b16584","value":"Downloading (…)okenizer_config.json: 100%"}},"5634196dc0bd4424843e7d721d28362a":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_7cb4cafea7e54229aad82870395b9301","max":2537,"min":0,"orientation":"horizontal","style":"IPY_MODEL_d9cf0cb2ee1d441089e6ac20d1e54a73","value":2537}},"c72e22a6ab40424286a185ac758a225e":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_78a272e0eac84a749b76b020b224e2c7","placeholder":"​","style":"IPY_MODEL_2bcc9f1090bb4ecbb47d42520d4d6c74","value":" 2.54k/2.54k [00:00&lt;00:00, 43.2kB/s]"}},"c9ca5157a2e44cbaa02cb0314ac8e8bf":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"048af3db70184a679800034200d93952":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"52380821e49d448887f932add9b16584":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7cb4cafea7e54229aad82870395b9301":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d9cf0cb2ee1d441089e6ac20d1e54a73":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"78a272e0eac84a749b76b020b224e2c7":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2bcc9f1090bb4ecbb47d42520d4d6c74":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6a38a48eb5034c049d0fa0dd162fde42":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_d611b1bef9aa412cbf6dd0becf687668","IPY_MODEL_252e6364186e4e58bba4176dfeaa228b","IPY_MODEL_4ce56b53ad74491786d3a4b82a09c542"],"layout":"IPY_MODEL_aa87ddf3f3534242b76614eceb45dc82"}},"d611b1bef9aa412cbf6dd0becf687668":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_54a0814e93494f0587bd9442442d60e5","placeholder":"​","style":"IPY_MODEL_94fc372b752744acb1531d8b1f1d02b5","value":"Downloading (…)&quot;spiece.model&quot;;: 100%"}},"252e6364186e4e58bba4176dfeaa228b":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_f4291e0fb592452da1534aca02b5b243","max":791656,"min":0,"orientation":"horizontal","style":"IPY_MODEL_6eb57a003ef04f859ee2830509bfbf93","value":791656}},"4ce56b53ad74491786d3a4b82a09c542":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ef1e797b056e450dac1978e5f4efe1f7","placeholder":"​","style":"IPY_MODEL_452c422624844f0c8110d152a9104c9f","value":" 792k/792k [00:00&lt;00:00, 6.38MB/s]"}},"aa87ddf3f3534242b76614eceb45dc82":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"54a0814e93494f0587bd9442442d60e5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"94fc372b752744acb1531d8b1f1d02b5":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f4291e0fb592452da1534aca02b5b243":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6eb57a003ef04f859ee2830509bfbf93":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"ef1e797b056e450dac1978e5f4efe1f7":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"452c422624844f0c8110d152a9104c9f":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8b43122390244756b58b00a4f9f1c688":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_17c157997bef447eb088bebf861f9bbe","IPY_MODEL_96a48ca37c2d43fa9dfb4c3c1304853d","IPY_MODEL_ce990df56c0940fc82cf14f707f8e3c7"],"layout":"IPY_MODEL_4c52f383bef64bb8b7a149ad13f2940b"}},"17c157997bef447eb088bebf861f9bbe":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_25ec8aec80864a329cf638caa132a817","placeholder":"​","style":"IPY_MODEL_5b4ae3431a0843158e741f442f113d23","value":"Downloading (…)/main/tokenizer.json: 100%"}},"96a48ca37c2d43fa9dfb4c3c1304853d":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_96b40bca89974400804e5e1f5137bd5d","max":2424064,"min":0,"orientation":"horizontal","style":"IPY_MODEL_8ef713aee0aa4684850f15b779a4c8d1","value":2424064}},"ce990df56c0940fc82cf14f707f8e3c7":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d9ee9576b8ef4788ba652a875584afc3","placeholder":"​","style":"IPY_MODEL_c0ffdbf9221546d09683913904bc2b21","value":" 2.42M/2.42M [00:00&lt;00:00, 11.2MB/s]"}},"4c52f383bef64bb8b7a149ad13f2940b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"25ec8aec80864a329cf638caa132a817":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5b4ae3431a0843158e741f442f113d23":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"96b40bca89974400804e5e1f5137bd5d":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8ef713aee0aa4684850f15b779a4c8d1":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"d9ee9576b8ef4788ba652a875584afc3":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c0ffdbf9221546d09683913904bc2b21":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3b408f071d0c400d9997aeeedb3c55bb":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b5a3a5676b634e3f9b845ea4107c5904","IPY_MODEL_b4392fe0efc34fad87c768ba499d5d97","IPY_MODEL_4ea08493d1024dd4b26ea6f8729942ef"],"layout":"IPY_MODEL_602780d3231f4f688305e405040f08dd"}},"b5a3a5676b634e3f9b845ea4107c5904":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3852f79010154f9c89278aacf1ce0b60","placeholder":"​","style":"IPY_MODEL_d96da6f789e74166841bd8f2cab5bfdc","value":"Downloading (…)cial_tokens_map.json: 100%"}},"b4392fe0efc34fad87c768ba499d5d97":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_ebf96cf2dcef46faa724639b045a8f3e","max":2201,"min":0,"orientation":"horizontal","style":"IPY_MODEL_28bb6d7812704dc3b1edcd519d78ead7","value":2201}},"4ea08493d1024dd4b26ea6f8729942ef":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_83f17383616d4ff0b73f0f7fa198d8fb","placeholder":"​","style":"IPY_MODEL_c11dfd5356d54640ad384937470f2730","value":" 2.20k/2.20k [00:00&lt;00:00, 25.7kB/s]"}},"602780d3231f4f688305e405040f08dd":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3852f79010154f9c89278aacf1ce0b60":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d96da6f789e74166841bd8f2cab5bfdc":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ebf96cf2dcef46faa724639b045a8f3e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"28bb6d7812704dc3b1edcd519d78ead7":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"83f17383616d4ff0b73f0f7fa198d8fb":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c11dfd5356d54640ad384937470f2730":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1d866806e15243a2a6b9e2896ab4b080":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_04e36c5f9ad54eb6a65d67a7f3fdc1fe","IPY_MODEL_fe942481952145bca2b065fdffc57a6e","IPY_MODEL_10fbb2a9d59f4ff7b93a861bd23ec11b"],"layout":"IPY_MODEL_50247834284941eea259a5459f873d78"}},"04e36c5f9ad54eb6a65d67a7f3fdc1fe":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e39fa7c808b74eca8d36cb520a88f9e9","placeholder":"​","style":"IPY_MODEL_af4350eb2fc448b4a1b5622bdae46f2b","value":"Downloading (…)lve/main/config.json: 100%"}},"fe942481952145bca2b065fdffc57a6e":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_5f0f327f5ae840539f5965141a2699b8","max":1404,"min":0,"orientation":"horizontal","style":"IPY_MODEL_9bac8da88a2f42cc8d78d614b4ed8c4b","value":1404}},"10fbb2a9d59f4ff7b93a861bd23ec11b":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_35c19f99eb06497d9de7cbf22bc3e470","placeholder":"​","style":"IPY_MODEL_db017986b1de4a60b4a8580520bda5b1","value":" 1.40k/1.40k [00:00&lt;00:00, 20.4kB/s]"}},"50247834284941eea259a5459f873d78":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e39fa7c808b74eca8d36cb520a88f9e9":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"af4350eb2fc448b4a1b5622bdae46f2b":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5f0f327f5ae840539f5965141a2699b8":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9bac8da88a2f42cc8d78d614b4ed8c4b":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"35c19f99eb06497d9de7cbf22bc3e470":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"db017986b1de4a60b4a8580520bda5b1":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"085045127f87427db73a60ef1e327611":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_53ad0ff2b0ab46b9bac26bb2151f7a15","IPY_MODEL_ff0432bc9c734568a3f3dd05aa7120ad","IPY_MODEL_ae9a1c87abfe47bb9910952d21816dd4"],"layout":"IPY_MODEL_631fb86b84694e71b4e07ed87814ecbf"}},"53ad0ff2b0ab46b9bac26bb2151f7a15":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9ea753414ae9483c897b5adfb20af8d4","placeholder":"​","style":"IPY_MODEL_bdde8de9755c44d6962b548c37d9c927","value":"Downloading (…)&quot;pytorch_model.bin&quot;;: 100%"}},"ff0432bc9c734568a3f3dd05aa7120ad":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a1085e971cf24f19b8e7f9f2483bb121","max":990402637,"min":0,"orientation":"horizontal","style":"IPY_MODEL_7b0f22568b804917822961916976b5a2","value":990402637}},"ae9a1c87abfe47bb9910952d21816dd4":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_22f612f85b3e405c9d7a8b6563bd41fb","placeholder":"​","style":"IPY_MODEL_b744c050c4484789894483a0c55275a5","value":" 990M/990M [00:11&lt;00:00, 87.8MB/s]"}},"631fb86b84694e71b4e07ed87814ecbf":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9ea753414ae9483c897b5adfb20af8d4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bdde8de9755c44d6962b548c37d9c927":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a1085e971cf24f19b8e7f9f2483bb121":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7b0f22568b804917822961916976b5a2":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"22f612f85b3e405c9d7a8b6563bd41fb":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b744c050c4484789894483a0c55275a5":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"af73747a621844c7a0560a1527a76c9d":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_7288441ae4d64cc89a674627c154512d","IPY_MODEL_91d82b64e46a4a6b8d706e12d6148aae","IPY_MODEL_89241d98f79640b39d443848c6e58007"],"layout":"IPY_MODEL_a983652b826543048fb2ef76e9411246"}},"7288441ae4d64cc89a674627c154512d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_348e436208d64851a731e120efd00038","placeholder":"​","style":"IPY_MODEL_4df5440f92674ad09462e65c90b3ddb5","value":"Downloading (…)neration_config.json: 100%"}},"91d82b64e46a4a6b8d706e12d6148aae":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_f7c349857ab24d51bcc5e718ec9812aa","max":147,"min":0,"orientation":"horizontal","style":"IPY_MODEL_9ae246e32e3544d19cf057573ff02b30","value":147}},"89241d98f79640b39d443848c6e58007":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_8897f4dfcbe94bf99e0af6222abe0298","placeholder":"​","style":"IPY_MODEL_82538e6c3518482aa1ad2357685ce2f6","value":" 147/147 [00:00&lt;00:00, 5.54kB/s]"}},"a983652b826543048fb2ef76e9411246":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"348e436208d64851a731e120efd00038":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4df5440f92674ad09462e65c90b3ddb5":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f7c349857ab24d51bcc5e718ec9812aa":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9ae246e32e3544d19cf057573ff02b30":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"8897f4dfcbe94bf99e0af6222abe0298":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"82538e6c3518482aa1ad2357685ce2f6":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","source":["# Instalacja pakietów"],"metadata":{"id":"ZXsOR6oJOJbd"}},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"8l0hzptKNiZS","executionInfo":{"status":"ok","timestamp":1676655657013,"user_tz":-60,"elapsed":25722,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"outputId":"e4743567-b234-4fae-8159-c51d9a68b430"},"outputs":[{"output_type":"stream","name":"stdout","text":["Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n","Collecting transformers\n","  Downloading transformers-4.26.1-py3-none-any.whl (6.3 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.3/6.3 MB\u001b[0m \u001b[31m42.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting datasets\n","  Downloading datasets-2.9.0-py3-none-any.whl (462 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m462.8/462.8 KB\u001b[0m \u001b[31m29.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: torch in /usr/local/lib/python3.8/dist-packages (1.13.1+cu116)\n","Collecting sentencepiece\n","  Downloading sentencepiece-0.1.97-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m32.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.8/dist-packages (from transformers) (4.64.1)\n","Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (23.0)\n","Collecting huggingface-hub<1.0,>=0.11.0\n","  Downloading huggingface_hub-0.12.1-py3-none-any.whl (190 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.3/190.3 KB\u001b[0m \u001b[31m15.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers) (3.9.0)\n","Collecting tokenizers!=0.11.3,<0.14,>=0.11.1\n","  Downloading tokenizers-0.13.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.6 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.6/7.6 MB\u001b[0m \u001b[31m27.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from transformers) (2.25.1)\n","Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (1.21.6)\n","Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (2022.6.2)\n","Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (6.0)\n","Collecting multiprocess\n","  Downloading multiprocess-0.70.14-py38-none-any.whl (132 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m132.0/132.0 KB\u001b[0m \u001b[31m5.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.8/dist-packages (from datasets) (2023.1.0)\n","Requirement already satisfied: aiohttp in /usr/local/lib/python3.8/dist-packages (from datasets) (3.8.4)\n","Requirement already satisfied: pandas in /usr/local/lib/python3.8/dist-packages (from datasets) (1.3.5)\n","Requirement already satisfied: dill<0.3.7 in /usr/local/lib/python3.8/dist-packages (from datasets) (0.3.6)\n","Collecting responses<0.19\n","  Downloading responses-0.18.0-py3-none-any.whl (38 kB)\n","Requirement already satisfied: pyarrow>=6.0.0 in /usr/local/lib/python3.8/dist-packages (from datasets) (9.0.0)\n","Collecting xxhash\n","  Downloading xxhash-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (213 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m213.0/213.0 KB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: typing-extensions in /usr/local/lib/python3.8/dist-packages (from torch) (4.5.0)\n","Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (4.0.2)\n","Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.1)\n","Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (22.2.0)\n","Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.8.2)\n","Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (6.0.4)\n","Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.3)\n","Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (3.0.1)\n","Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2.10)\n","Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (1.24.3)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2022.12.7)\n","Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (4.0.0)\n","Collecting urllib3<1.27,>=1.21.1\n","  Downloading urllib3-1.26.14-py2.py3-none-any.whl (140 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m140.6/140.6 KB\u001b[0m \u001b[31m6.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas->datasets) (2022.7.1)\n","Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas->datasets) (2.8.2)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil>=2.7.3->pandas->datasets) (1.15.0)\n","Installing collected packages: tokenizers, sentencepiece, xxhash, urllib3, multiprocess, responses, huggingface-hub, transformers, datasets\n","  Attempting uninstall: urllib3\n","    Found existing installation: urllib3 1.24.3\n","    Uninstalling urllib3-1.24.3:\n","      Successfully uninstalled urllib3-1.24.3\n","Successfully installed datasets-2.9.0 huggingface-hub-0.12.1 multiprocess-0.70.14 responses-0.18.0 sentencepiece-0.1.97 tokenizers-0.13.2 transformers-4.26.1 urllib3-1.26.14 xxhash-3.2.0\n"]}],"source":["!pip install transformers datasets torch sentencepiece"]},{"cell_type":"markdown","source":["# Załadowanie datasetu"],"metadata":{"id":"dhN0rmb5Oi3d"}},{"cell_type":"code","source":["from datasets import load_dataset"],"metadata":{"id":"tnaDkwZ2Pbnn","executionInfo":{"status":"ok","timestamp":1676655657691,"user_tz":-60,"elapsed":683,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}}},"execution_count":2,"outputs":[]},{"cell_type":"code","source":["dataset = load_dataset(\"sms_spam\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":232,"referenced_widgets":["b6eded70136441759cc5ac10b00acb8f","c5f3c896b6534c178ae822c32a8e00b3","fe35cc2de5b840e49d9d9c5fec869656","532adf3b2f7a4b75a095e9455351553c","646379eb229040dcb09aa0f03d6f3f26","5ee0acafd6944686b9ef3dedabed6aef","e3f4a5f8f4864b0d8357aa29d2b8802f","9152a884650a486f9275278678061eec","9e9b5088b6054ba2a48c6b7872a2c383","f222becd734644c4ae631d432e1b3156","510afccfa74748c4bd69a10dc53e3b70","e7db60e680f04318a706fcb1fe7dbcb7","101d73ffbf8c47af9f62103648edc548","5a5500aa9fd84381be7676f570665d7a","76c1dca21a3c4c3c91769c48a4428ba3","cee9b746e101471cb5808e2415091f38","292d1d346d9246f99b87709fac58b0f0","94ddbe64d035427d943606085a697f23","8f9eb5f919144f74856d354642cff880","176ebdce409b49d59423292931d8a5df","a35c33cb3774427f8e739a7b1c948f4e","ea9bb434237643ab8bb4e4d6f884ba67","b1810e9efa944f08ba50be015456287f","76abc09e7e794eef9ffc3081b2ec2da6","2fbff5b358734b579a1b1d471934150d","4a0b0fc9717b4fc6bf5ad887785dec90","af367d23351544f187286a069b469e89","dd56ef64cae84c3f8dcb52b86bfeeede","107cf57f7ba34a7b82a99035098cd60a","d3f4f7df0b8f41b7a11bf9ca7a2c02d1","f0f50dea946240a79adce466c2810c5a","0d06ef4561694b2e8a5c495f55f01320","d8f9577b8ef8418597e3e327ad29e119","97a4213ad47f47aaae8dce4947c541d5","4ae43dc6645b4e7abfd5018b40b6e1cf","1f2721589fd549269dd8c04fb00ac0b1","cfe42f1c148c4507b8bed4eade47d2cd","a91105129f0a4d0daf1f92beee7b696b","ec41724b9add4285bc5d04715505675b","a1ef5bde067c4250b5d5b74ce8b2945d","79810b4bd66442aba12328e39a00e438","5f0c3fbda713440d80ec4d0654b5892d","78e6dfa135804ed785e0a809356c5112","5fe984999000415b8770abe7c6d65bfc","f53b16108c93439d9943b35c28d57988","8a180e9d3cef4d58a5b05509edb39ec4","d97533a9d431471c898cbccb93202900","235bdfd6aee042a587647925981a1fd2","e081d46be72b4d638cf3cc7be14ac651","0154f8d5e1ef4584b2def2e456179951","d2d1b1c19e8e4b18af10bffe9c3a60f5","40b5e64a2d6743cbbb6b3dca004836b9","e9a816019ddc4c34a94059381da2b7b0","e0a987f764274098ace0ce3900e37c5e","e5b7397c80a44e66863276ed6e443dd9","594b168dc8f24064afca0a0353195c35","b641fc8e31514c2e9c9453ba3669575c","c575ee9e9a4846669a70be243ec8a228","48ac88309d404e1cb83e660a8b344ac6","0e48acebc7b04eb2a1b55a0d852ef7b3","edcf26a3835b4db6b3fbfd160b61aa2e","3eaee86b67704d3da78285c7881cff01","167ad19c7da2466db0528865578fda87","5c384622a46e4252a123378d29bc7640","a60f375b3eee415388d742c619966776","39f30b2919dc4ac0b6e3c8d54999a628"]},"id":"cCiAuRqrOkvV","executionInfo":{"status":"ok","timestamp":1676655660188,"user_tz":-60,"elapsed":2500,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"outputId":"05d2045e-0777-4174-b3b9-d9fe91ea990f"},"execution_count":3,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading builder script:   0%|          | 0.00/3.21k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"b6eded70136441759cc5ac10b00acb8f"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading metadata:   0%|          | 0.00/1.69k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"e7db60e680f04318a706fcb1fe7dbcb7"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading readme:   0%|          | 0.00/4.87k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"b1810e9efa944f08ba50be015456287f"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Downloading and preparing dataset sms_spam/plain_text to /root/.cache/huggingface/datasets/sms_spam/plain_text/1.0.0/53f051d3b5f62d99d61792c91acefe4f1577ad3e4c216fb0ad39e30b9f20019c...\n"]},{"output_type":"display_data","data":{"text/plain":["Downloading data:   0%|          | 0.00/203k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"97a4213ad47f47aaae8dce4947c541d5"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Generating train split:   0%|          | 0/5574 [00:00<?, ? examples/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"f53b16108c93439d9943b35c28d57988"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Dataset sms_spam downloaded and prepared to /root/.cache/huggingface/datasets/sms_spam/plain_text/1.0.0/53f051d3b5f62d99d61792c91acefe4f1577ad3e4c216fb0ad39e30b9f20019c. Subsequent calls will reuse this data.\n"]},{"output_type":"display_data","data":{"text/plain":["  0%|          | 0/1 [00:00<?, ?it/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"594b168dc8f24064afca0a0353195c35"}},"metadata":{}}]},{"cell_type":"code","source":["dataset['train'][0]"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"JKFHPko3OnAV","executionInfo":{"status":"ok","timestamp":1676655660189,"user_tz":-60,"elapsed":5,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"outputId":"43a7c656-f0bf-4f82-b03e-ebd0b79f377a"},"execution_count":4,"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\\n',\n"," 'label': 0}"]},"metadata":{},"execution_count":4}]},{"cell_type":"markdown","source":["# Przygotowanie datasetu"],"metadata":{"id":"l140vJrgYxPr"}},{"cell_type":"code","source":["parsed_dataset = []\n","\n","for row in dataset['train']:\n","  text = row['sms'].replace(\"\\n\", \"\")\n","  new_row = {}\n","  new_row['sms'] = text\n","  if row['label'] == 0:\n","    new_row['label'] = \"False\"\n","  else:\n","    new_row['label'] = \"True\"\n","  parsed_dataset.append(new_row)\n","\n","parsed_dataset[0]"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"1boUF-YiY3_y","executionInfo":{"status":"ok","timestamp":1676655661173,"user_tz":-60,"elapsed":987,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"outputId":"add4c55f-079c-4855-d13c-7bde48bedad4"},"execution_count":5,"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...',\n"," 'label': 'False'}"]},"metadata":{},"execution_count":5}]},{"cell_type":"markdown","source":["# Tokenizer FLAN-T5"],"metadata":{"id":"O-J-jBDxPJcn"}},{"cell_type":"code","source":["from transformers import AutoTokenizer"],"metadata":{"id":"P23AYPX1PZ6g","executionInfo":{"status":"ok","timestamp":1676655663313,"user_tz":-60,"elapsed":2143,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}}},"execution_count":6,"outputs":[]},{"cell_type":"code","source":["tokenizer = AutoTokenizer.from_pretrained('google/flan-t5-base')"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":145,"referenced_widgets":["964a243415be46aba8b943310ab54d4f","d9ccbf0d26014aa584f3e2d0d0e64a68","5634196dc0bd4424843e7d721d28362a","c72e22a6ab40424286a185ac758a225e","c9ca5157a2e44cbaa02cb0314ac8e8bf","048af3db70184a679800034200d93952","52380821e49d448887f932add9b16584","7cb4cafea7e54229aad82870395b9301","d9cf0cb2ee1d441089e6ac20d1e54a73","78a272e0eac84a749b76b020b224e2c7","2bcc9f1090bb4ecbb47d42520d4d6c74","6a38a48eb5034c049d0fa0dd162fde42","d611b1bef9aa412cbf6dd0becf687668","252e6364186e4e58bba4176dfeaa228b","4ce56b53ad74491786d3a4b82a09c542","aa87ddf3f3534242b76614eceb45dc82","54a0814e93494f0587bd9442442d60e5","94fc372b752744acb1531d8b1f1d02b5","f4291e0fb592452da1534aca02b5b243","6eb57a003ef04f859ee2830509bfbf93","ef1e797b056e450dac1978e5f4efe1f7","452c422624844f0c8110d152a9104c9f","8b43122390244756b58b00a4f9f1c688","17c157997bef447eb088bebf861f9bbe","96a48ca37c2d43fa9dfb4c3c1304853d","ce990df56c0940fc82cf14f707f8e3c7","4c52f383bef64bb8b7a149ad13f2940b","25ec8aec80864a329cf638caa132a817","5b4ae3431a0843158e741f442f113d23","96b40bca89974400804e5e1f5137bd5d","8ef713aee0aa4684850f15b779a4c8d1","d9ee9576b8ef4788ba652a875584afc3","c0ffdbf9221546d09683913904bc2b21","3b408f071d0c400d9997aeeedb3c55bb","b5a3a5676b634e3f9b845ea4107c5904","b4392fe0efc34fad87c768ba499d5d97","4ea08493d1024dd4b26ea6f8729942ef","602780d3231f4f688305e405040f08dd","3852f79010154f9c89278aacf1ce0b60","d96da6f789e74166841bd8f2cab5bfdc","ebf96cf2dcef46faa724639b045a8f3e","28bb6d7812704dc3b1edcd519d78ead7","83f17383616d4ff0b73f0f7fa198d8fb","c11dfd5356d54640ad384937470f2730"]},"id":"q5Jz0E_oPMBr","executionInfo":{"status":"ok","timestamp":1676655665494,"user_tz":-60,"elapsed":2191,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"outputId":"63846fa5-30f2-43bb-b10f-4a2df75f8f1e"},"execution_count":7,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading (…)okenizer_config.json:   0%|          | 0.00/2.54k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"964a243415be46aba8b943310ab54d4f"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)\"spiece.model\";:   0%|          | 0.00/792k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6a38a48eb5034c049d0fa0dd162fde42"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)/main/tokenizer.json:   0%|          | 0.00/2.42M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"8b43122390244756b58b00a4f9f1c688"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)cial_tokens_map.json:   0%|          | 0.00/2.20k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"3b408f071d0c400d9997aeeedb3c55bb"}},"metadata":{}}]},{"cell_type":"code","source":["sms = parsed_dataset[0]['sms']\n","print('Original: ', sms)\n","print('Tokenized: ', tokenizer.tokenize(sms))\n","print('Token IDs: ', tokenizer.convert_tokens_to_ids(tokenizer.tokenize(sms)))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"dfxJQpoePsvI","executionInfo":{"status":"ok","timestamp":1676655665494,"user_tz":-60,"elapsed":15,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"outputId":"d4da4d86-1f3b-4d6c-bd8a-532230c3539b"},"execution_count":8,"outputs":[{"output_type":"stream","name":"stdout","text":["Original:  Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\n","Tokenized:  ['▁Go', '▁until', '▁jur', 'ong', '▁point', ',', '▁crazy', '.', '.', '▁Available', '▁only', '▁in', '▁bug', 'is', '▁', 'n', '▁great', '▁world', '▁la', '▁', 'e', '▁buffet', '...', '▁Cine', '▁there', '▁got', '▁', 'a', 'more', '▁wa', 't', '...']\n","Token IDs:  [1263, 552, 10081, 2444, 500, 6, 6139, 5, 5, 8144, 163, 16, 8143, 159, 3, 29, 248, 296, 50, 3, 15, 15385, 233, 17270, 132, 530, 3, 9, 3706, 8036, 17, 233]\n"]}]},{"cell_type":"markdown","source":["# Few shot learning"],"metadata":{"id":"UpluhM8cU5Ir"}},{"cell_type":"code","source":["print(parsed_dataset[0]) #0\n","print(parsed_dataset[123]) #1\n","print(parsed_dataset[2000]) #0\n","print(parsed_dataset[3002]) #1"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"7uNUkixPU85O","executionInfo":{"status":"ok","timestamp":1676655665495,"user_tz":-60,"elapsed":13,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"outputId":"6eda01c2-b3ad-4f08-919e-2a6e8b6beb31"},"execution_count":9,"outputs":[{"output_type":"stream","name":"stdout","text":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...', 'label': 'False'}\n","{'sms': 'Todays Voda numbers ending 7548 are selected to receive a $350 award. If you have a match please call 08712300220 quoting claim code 4041 standard rates app', 'label': 'True'}\n","{'sms': \"LMAO where's your fish memory when I need it?\", 'label': 'False'}\n","{'sms': 'This message is free. Welcome to the new & improved Sex & Dogging club! To unsubscribe from this service reply STOP. msgs@150p 18+only', 'label': 'True'}\n"]}]},{"cell_type":"code","source":["non_spam_1 = \"The text message: \" + parsed_dataset[0]['sms'] + \"\\nIs this sms spam? Ans: False\\n\\n\"\n","spam_1 = \"The text message: \" + parsed_dataset[123]['sms'] + \"\\nIs this sms spam? Ans: True\\n\\n\"\n","non_spam_2 = \"The text message: \" + parsed_dataset[2000]['sms'] + \"\\nIs this sms spam? Ans: False\\n\\n\"\n","spam_2 = \"The text message: \" + parsed_dataset[3002]['sms'] + \"\\nIs this sms spam? Ans: True\\n\\n\"\n","\n","few_shot_prefix = non_spam_1 + spam_1 + non_spam_2 + spam_2 + \"The text message: \"\n","print(few_shot_prefix)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"lj0issBznZfK","executionInfo":{"status":"ok","timestamp":1676655665495,"user_tz":-60,"elapsed":11,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"outputId":"84d05249-8a44-4217-d08e-d4aa86a73063"},"execution_count":10,"outputs":[{"output_type":"stream","name":"stdout","text":["The text message: Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\n","Is this sms spam? Ans: False\n","\n","The text message: Todays Voda numbers ending 7548 are selected to receive a $350 award. If you have a match please call 08712300220 quoting claim code 4041 standard rates app\n","Is this sms spam? Ans: True\n","\n","The text message: LMAO where's your fish memory when I need it?\n","Is this sms spam? Ans: False\n","\n","The text message: This message is free. Welcome to the new & improved Sex & Dogging club! To unsubscribe from this service reply STOP. msgs@150p 18+only\n","Is this sms spam? Ans: True\n","\n","The text message: \n"]}]},{"cell_type":"markdown","source":["# Load FLAN-T5 model"],"metadata":{"id":"okTx_ynMV0rH"}},{"cell_type":"code","source":["from transformers import AutoModelForSeq2SeqLM"],"metadata":{"id":"Eu-7Eed8WgN0","executionInfo":{"status":"ok","timestamp":1676655665496,"user_tz":-60,"elapsed":10,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}}},"execution_count":11,"outputs":[]},{"cell_type":"code","source":["model = AutoModelForSeq2SeqLM.from_pretrained('google/flan-t5-base')\n","\n","model.cuda()"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000,"referenced_widgets":["1d866806e15243a2a6b9e2896ab4b080","04e36c5f9ad54eb6a65d67a7f3fdc1fe","fe942481952145bca2b065fdffc57a6e","10fbb2a9d59f4ff7b93a861bd23ec11b","50247834284941eea259a5459f873d78","e39fa7c808b74eca8d36cb520a88f9e9","af4350eb2fc448b4a1b5622bdae46f2b","5f0f327f5ae840539f5965141a2699b8","9bac8da88a2f42cc8d78d614b4ed8c4b","35c19f99eb06497d9de7cbf22bc3e470","db017986b1de4a60b4a8580520bda5b1","085045127f87427db73a60ef1e327611","53ad0ff2b0ab46b9bac26bb2151f7a15","ff0432bc9c734568a3f3dd05aa7120ad","ae9a1c87abfe47bb9910952d21816dd4","631fb86b84694e71b4e07ed87814ecbf","9ea753414ae9483c897b5adfb20af8d4","bdde8de9755c44d6962b548c37d9c927","a1085e971cf24f19b8e7f9f2483bb121","7b0f22568b804917822961916976b5a2","22f612f85b3e405c9d7a8b6563bd41fb","b744c050c4484789894483a0c55275a5","af73747a621844c7a0560a1527a76c9d","7288441ae4d64cc89a674627c154512d","91d82b64e46a4a6b8d706e12d6148aae","89241d98f79640b39d443848c6e58007","a983652b826543048fb2ef76e9411246","348e436208d64851a731e120efd00038","4df5440f92674ad09462e65c90b3ddb5","f7c349857ab24d51bcc5e718ec9812aa","9ae246e32e3544d19cf057573ff02b30","8897f4dfcbe94bf99e0af6222abe0298","82538e6c3518482aa1ad2357685ce2f6"]},"id":"JKv9O8kfV2zZ","executionInfo":{"status":"ok","timestamp":1676655691072,"user_tz":-60,"elapsed":25585,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"outputId":"171d7691-dd21-4661-a11f-d1a7fe31cc5d"},"execution_count":12,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading (…)lve/main/config.json:   0%|          | 0.00/1.40k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"1d866806e15243a2a6b9e2896ab4b080"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)\"pytorch_model.bin\";:   0%|          | 0.00/990M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"085045127f87427db73a60ef1e327611"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)neration_config.json:   0%|          | 0.00/147 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"af73747a621844c7a0560a1527a76c9d"}},"metadata":{}},{"output_type":"execute_result","data":{"text/plain":["T5ForConditionalGeneration(\n","  (shared): Embedding(32128, 768)\n","  (encoder): T5Stack(\n","    (embed_tokens): Embedding(32128, 768)\n","    (block): ModuleList(\n","      (0): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","              (relative_attention_bias): Embedding(32, 12)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (1): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (2): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (3): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (4): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (5): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (6): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (7): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (8): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (9): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (10): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (11): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","    )\n","    (final_layer_norm): T5LayerNorm()\n","    (dropout): Dropout(p=0.1, inplace=False)\n","  )\n","  (decoder): T5Stack(\n","    (embed_tokens): Embedding(32128, 768)\n","    (block): ModuleList(\n","      (0): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","              (relative_attention_bias): Embedding(32, 12)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (1): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (2): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (3): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (4): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (5): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (6): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (7): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (8): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (9): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (10): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (11): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=768, out_features=768, bias=False)\n","              (k): Linear(in_features=768, out_features=768, bias=False)\n","              (v): Linear(in_features=768, out_features=768, bias=False)\n","              (o): Linear(in_features=768, out_features=768, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=768, out_features=2048, bias=False)\n","              (wi_1): Linear(in_features=768, out_features=2048, bias=False)\n","              (wo): Linear(in_features=2048, out_features=768, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","    )\n","    (final_layer_norm): T5LayerNorm()\n","    (dropout): Dropout(p=0.1, inplace=False)\n","  )\n","  (lm_head): Linear(in_features=768, out_features=32128, bias=False)\n",")"]},"metadata":{},"execution_count":12}]},{"cell_type":"markdown","source":["# Helper functions"],"metadata":{"id":"F_SDAwxoawDy"}},{"cell_type":"code","source":["import torch"],"metadata":{"id":"rdWMg_KJZEZH","executionInfo":{"status":"ok","timestamp":1676655691073,"user_tz":-60,"elapsed":8,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}}},"execution_count":13,"outputs":[]},{"cell_type":"code","source":["def calculate_accuracy(preds, target):\n","  results_ok = 0.0\n","  results_false = 0.0\n","\n","  for idx, pred in enumerate(preds):\n","    if pred == target[idx]:\n","      results_ok += 1.0\n","    else:\n","      results_false += 1.0\n","\n","  return results_ok / (results_ok + results_false)"],"metadata":{"id":"FzUi8908ax61","executionInfo":{"status":"ok","timestamp":1676655691074,"user_tz":-60,"elapsed":8,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}}},"execution_count":14,"outputs":[]},{"cell_type":"code","source":["if torch.cuda.is_available():     \n","    device = torch.device(\"cuda\")\n","\n","    print('There are %d GPU(s) available.' % torch.cuda.device_count())\n","    print('We will use the GPU:', torch.cuda.get_device_name(0))\n","\n","else:\n","    print('No GPU available, using the CPU instead.')\n","    device = torch.device(\"cpu\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"86i7iRmtW-6L","executionInfo":{"status":"ok","timestamp":1676655691074,"user_tz":-60,"elapsed":8,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"outputId":"d0f160f4-a67c-4269-d22c-db44fe8c9457"},"execution_count":15,"outputs":[{"output_type":"stream","name":"stdout","text":["There are 1 GPU(s) available.\n","We will use the GPU: Tesla T4\n"]}]},{"cell_type":"markdown","source":["# Predykcja"],"metadata":{"id":"H_YI3bS3VHQE"}},{"cell_type":"code","source":["parsed_dataset = parsed_dataset[1:123] + parsed_dataset[124:2000] + parsed_dataset[2001:3002] + parsed_dataset[3003:]\n","predictions = []\n","expected = []\n","\n","for row in parsed_dataset:\n","  input_text = few_shot_prefix + row['sms'] + \"\\nIs this sms spam? Ans: \"\n","  input_ids = tokenizer(input_text, return_tensors=\"pt\").input_ids.to(device)\n","\n","  generated_ids = model.generate(input_ids, do_sample=True, temperature=0.9, max_length=300)\n","  generated_text = tokenizer.decode(generated_ids[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)\n","  \n","  predictions.append(generated_text)\n","  expected.append(row['label'])\n","\n","acc = calculate_accuracy(predictions, expected)\n","print(acc)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"vunLGuBGVGmh","executionInfo":{"status":"ok","timestamp":1676656276714,"user_tz":-60,"elapsed":384554,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"outputId":"98047a20-86db-4d62-bd76-a8daaca995d6"},"execution_count":17,"outputs":[{"output_type":"stream","name":"stdout","text":["0.28404599353215954\n"]}]},{"cell_type":"code","source":["print(\"Sample prediction: {}, expected: {}\".format(predictions[101], expected[101]))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"20WbYZmLaDl7","executionInfo":{"status":"ok","timestamp":1676656326544,"user_tz":-60,"elapsed":235,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"outputId":"70a99496-a105-4d1f-f713-c3c08aa48b1a"},"execution_count":18,"outputs":[{"output_type":"stream","name":"stdout","text":["Sample prediction: Yes, expected: False\n"]}]},{"cell_type":"markdown","source":["# MCC Score"],"metadata":{"id":"dLYc9WXz_B1o"}},{"cell_type":"code","source":["from sklearn.metrics import matthews_corrcoef\n","\n","print('Calculating Matthews Corr. Coef. for each batch...')\n","matthews = matthews_corrcoef(expected, predictions)    \n","print('Total MCC: %.3f' % matthews)"],"metadata":{"id":"hPEPpXXX_DXR","executionInfo":{"status":"ok","timestamp":1676656330154,"user_tz":-60,"elapsed":787,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"c970bf13-032f-4912-a83c-91276ca70465"},"execution_count":19,"outputs":[{"output_type":"stream","name":"stdout","text":["Calculating Matthews Corr. Coef. for each batch...\n","Total MCC: 0.006\n"]}]},{"cell_type":"markdown","source":["# Save model"],"metadata":{"id":"GPhCp068_Iwq"}},{"cell_type":"code","source":["from google.colab import drive\n","\n","drive.mount('/content/gdrive/', force_remount=True)\n","\n","output_dir = '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_5_FLAN-T5_Model'\n","print(\"Saving model to %s\" % output_dir)\n","\n","model_to_save = model.module if hasattr(model, 'module') else model\n","model_to_save.save_pretrained(output_dir)\n","tokenizer.save_pretrained(output_dir)"],"metadata":{"id":"avafCMoS_KDF","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1676656352075,"user_tz":-60,"elapsed":19468,"user":{"displayName":"Mike3 Studia3","userId":"17253749482577755833"}},"outputId":"649fa4ed-c4cf-44bd-9137-013045151068"},"execution_count":20,"outputs":[{"output_type":"stream","name":"stdout","text":["Mounted at /content/gdrive/\n","Saving model to /content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_5_FLAN-T5_Model\n"]},{"output_type":"execute_result","data":{"text/plain":["('/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_5_FLAN-T5_Model/tokenizer_config.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_5_FLAN-T5_Model/special_tokens_map.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_5_FLAN-T5_Model/spiece.model',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_5_FLAN-T5_Model/added_tokens.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_5_FLAN-T5_Model/tokenizer.json')"]},"metadata":{},"execution_count":20}]},{"cell_type":"markdown","source":["# Bibliografia\n","- https://huggingface.co/docs/transformers/main/en/model_doc/flan-t5\n","- https://mccormickml.com/2019/07/22/BERT-fine-tuning/#a1-saving--loading-fine-tuned-model\n","- https://huggingface.co/docs/transformers/model_doc/t5#training"],"metadata":{"id":"wHzm2_nDA6i-"}}]}
\ No newline at end of file
diff --git a/projekt/Ver_6_FLAN-T5_sms_spam.ipynb b/projekt/Ver_6_FLAN-T5_sms_spam.ipynb
new file mode 100644
index 0000000..733cac4
--- /dev/null
+++ b/projekt/Ver_6_FLAN-T5_sms_spam.ipynb
@@ -0,0 +1 @@
+{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"},"gpuClass":"standard","widgets":{"application/vnd.jupyter.widget-state+json":{"c9c5041c2d734e449a3f480d27c6930f":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_93c307f83f8345569c6d5eb992facb8f","IPY_MODEL_0b1c4594b3394af9bf6bdb1f5c21efde","IPY_MODEL_8acffba7f8d841bca237676946643b85"],"layout":"IPY_MODEL_6cd1d313107b458fa45e5872c9ce66f0"}},"93c307f83f8345569c6d5eb992facb8f":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d6a531cb1e5442398ad46a7555af15ff","placeholder":"​","style":"IPY_MODEL_a682581443054d6ab11c4445b59ebaf8","value":"Downloading builder script: 100%"}},"0b1c4594b3394af9bf6bdb1f5c21efde":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_7eca2a8a527c417794f7902354dff4fa","max":3208,"min":0,"orientation":"horizontal","style":"IPY_MODEL_44c053a9f38d42ff8b1a37e7892a87d2","value":3208}},"8acffba7f8d841bca237676946643b85":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_086a54cdf54a4631b1d8aa8c835f74a5","placeholder":"​","style":"IPY_MODEL_5334c04227694b0dbfca313b8b3c1b73","value":" 3.21k/3.21k [00:00&lt;00:00, 148kB/s]"}},"6cd1d313107b458fa45e5872c9ce66f0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d6a531cb1e5442398ad46a7555af15ff":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a682581443054d6ab11c4445b59ebaf8":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7eca2a8a527c417794f7902354dff4fa":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"44c053a9f38d42ff8b1a37e7892a87d2":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"086a54cdf54a4631b1d8aa8c835f74a5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5334c04227694b0dbfca313b8b3c1b73":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ed0315a7a3734a9096b646d6087113d5":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_63e57336638c43269033927fea082b5e","IPY_MODEL_5c74805203c149a28b39b8a6673b3226","IPY_MODEL_bb1356f3522c48779c6ef331fd0cb258"],"layout":"IPY_MODEL_a9764585f1304fac85a4647205a07248"}},"63e57336638c43269033927fea082b5e":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_fe9a18c0f645464687a47a598c29856c","placeholder":"​","style":"IPY_MODEL_4cac33b53d5c47bf9e2559a19d786ba7","value":"Downloading metadata: 100%"}},"5c74805203c149a28b39b8a6673b3226":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_c9c783ba8f2c47d2a63f6008c6c7ebf6","max":1687,"min":0,"orientation":"horizontal","style":"IPY_MODEL_97a86f253eed42059461528ac274ff5f","value":1687}},"bb1356f3522c48779c6ef331fd0cb258":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_db85eaf5afc64c258db557856084caa7","placeholder":"​","style":"IPY_MODEL_08e5985a87ce49469632b644a0be1e4a","value":" 1.69k/1.69k [00:00&lt;00:00, 113kB/s]"}},"a9764585f1304fac85a4647205a07248":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fe9a18c0f645464687a47a598c29856c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4cac33b53d5c47bf9e2559a19d786ba7":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c9c783ba8f2c47d2a63f6008c6c7ebf6":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"97a86f253eed42059461528ac274ff5f":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"db85eaf5afc64c258db557856084caa7":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"08e5985a87ce49469632b644a0be1e4a":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6705f6b0f9754c0ebc4cb0dc1a37ac94":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_3843c8652968472fb3bfb4ce97b7c45b","IPY_MODEL_a248a43d17ab4633b88a1621942d4757","IPY_MODEL_37c5cf19bd0646aa9ce53218326424f7"],"layout":"IPY_MODEL_f55517fb17e0463e8c8b01f358534914"}},"3843c8652968472fb3bfb4ce97b7c45b":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_fddbdaedf5654097942c38002a299e3f","placeholder":"​","style":"IPY_MODEL_10ed5de382e244d8a56c47c76d90748f","value":"Downloading readme: 100%"}},"a248a43d17ab4633b88a1621942d4757":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_e559dfd597ad454b8d8c2ee005901a35","max":4872,"min":0,"orientation":"horizontal","style":"IPY_MODEL_f48f84cd18ab43b39bf2ca69599ad8c7","value":4872}},"37c5cf19bd0646aa9ce53218326424f7":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_1c40bc5715ea448697ac53773d86237a","placeholder":"​","style":"IPY_MODEL_fc419546e84144349e9c533a8176d4e4","value":" 4.87k/4.87k [00:00&lt;00:00, 87.7kB/s]"}},"f55517fb17e0463e8c8b01f358534914":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fddbdaedf5654097942c38002a299e3f":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"10ed5de382e244d8a56c47c76d90748f":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"e559dfd597ad454b8d8c2ee005901a35":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f48f84cd18ab43b39bf2ca69599ad8c7":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1c40bc5715ea448697ac53773d86237a":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fc419546e84144349e9c533a8176d4e4":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9b3a088c8a7a49b496c6b2e5d2d474dd":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_a803fe9e84344cdfac171dc6343121e5","IPY_MODEL_12f4b7d3045340d99001a098e0600c64","IPY_MODEL_a92495c6264a4256b9d1c4dd207d9041"],"layout":"IPY_MODEL_c0f1d46ebbdc4bfe8d31985f71875962"}},"a803fe9e84344cdfac171dc6343121e5":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e2a740892ddf477ba0b404b1e7e2cd88","placeholder":"​","style":"IPY_MODEL_b4eb68bbd452409f8faf6f77093f84da","value":"Downloading data: 100%"}},"12f4b7d3045340d99001a098e0600c64":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_9f8452af363d4a0daf38a7edf7d0a540","max":203415,"min":0,"orientation":"horizontal","style":"IPY_MODEL_ec09103848594bc38e167a16a407cc04","value":203415}},"a92495c6264a4256b9d1c4dd207d9041":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5b8f226314054f1f893d922347c31055","placeholder":"​","style":"IPY_MODEL_9e4f882e90234dbca0702be76099be77","value":" 203k/203k [00:00&lt;00:00, 231kB/s]"}},"c0f1d46ebbdc4bfe8d31985f71875962":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e2a740892ddf477ba0b404b1e7e2cd88":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b4eb68bbd452409f8faf6f77093f84da":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9f8452af363d4a0daf38a7edf7d0a540":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ec09103848594bc38e167a16a407cc04":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"5b8f226314054f1f893d922347c31055":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9e4f882e90234dbca0702be76099be77":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ef20074dae184d8cb1ead4ce74c2df0b":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b4721cb7a4f1486384078edb29a219f5","IPY_MODEL_b8194c781462455ab6b975689d934be3","IPY_MODEL_4cafb17414cf4413b70b04381b3783ca"],"layout":"IPY_MODEL_70819e011c9a4e759758ec8b1e6116f4"}},"b4721cb7a4f1486384078edb29a219f5":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_bfd5db4a4c9f4a0a9165e3b3a34f6e32","placeholder":"​","style":"IPY_MODEL_50586a5a007f438d94dad66ad03ca1ba","value":"Generating train split:  96%"}},"b8194c781462455ab6b975689d934be3":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"","description_tooltip":null,"layout":"IPY_MODEL_7295282fb11d4c95aac18ce9047f4da2","max":5574,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1354813182d7427b87f56a8aafc06c34","value":5574}},"4cafb17414cf4413b70b04381b3783ca":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_eaadb5abf84d455994f5378aac2a293c","placeholder":"​","style":"IPY_MODEL_dc7f94f8535a4734a3cd9fc9cef5fc2e","value":" 5332/5574 [00:00&lt;00:00, 6106.61 examples/s]"}},"70819e011c9a4e759758ec8b1e6116f4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":"hidden","width":null}},"bfd5db4a4c9f4a0a9165e3b3a34f6e32":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"50586a5a007f438d94dad66ad03ca1ba":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7295282fb11d4c95aac18ce9047f4da2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1354813182d7427b87f56a8aafc06c34":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"eaadb5abf84d455994f5378aac2a293c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"dc7f94f8535a4734a3cd9fc9cef5fc2e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"e5985e951e854e4fbdc8983b57830209":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_14ecc67173cd4eb2a20d51f817a01a29","IPY_MODEL_bd66d5b4736647a89ec35d65d55f7556","IPY_MODEL_eece11a05e874005ae2532f8139490d0"],"layout":"IPY_MODEL_7ba99dc8284d436f83a341d914e3b226"}},"14ecc67173cd4eb2a20d51f817a01a29":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_91b51e4d897843648bc69a4dc684f4bf","placeholder":"​","style":"IPY_MODEL_def43747b4624cd693383277e8520e1a","value":"100%"}},"bd66d5b4736647a89ec35d65d55f7556":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_438240aff4aa4c30ab5dc8a1159f32be","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_12772e6a0d1d40e9934ba6c663896c17","value":1}},"eece11a05e874005ae2532f8139490d0":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_68911a228fc343bbb84fd7f6b53f5fb4","placeholder":"​","style":"IPY_MODEL_9de6fd732fbb4ff5b917f8250710d342","value":" 1/1 [00:00&lt;00:00, 10.37it/s]"}},"7ba99dc8284d436f83a341d914e3b226":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"91b51e4d897843648bc69a4dc684f4bf":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"def43747b4624cd693383277e8520e1a":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"438240aff4aa4c30ab5dc8a1159f32be":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"12772e6a0d1d40e9934ba6c663896c17":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"68911a228fc343bbb84fd7f6b53f5fb4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9de6fd732fbb4ff5b917f8250710d342":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ac64c83f91604136a8605a458cfdb558":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_a51333fd9ed341119a8517ead52bed95","IPY_MODEL_b975f63617a6419d826d5999c33dadcd","IPY_MODEL_71f6deb181d04ac495d8be7405a9daea"],"layout":"IPY_MODEL_d5d18a69055b496fb29c737c940c6a3d"}},"a51333fd9ed341119a8517ead52bed95":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_2225a24fcae94178a948e2f05d7499db","placeholder":"​","style":"IPY_MODEL_d57673cadfd245e49e339f780362d602","value":"Downloading (…)okenizer_config.json: 100%"}},"b975f63617a6419d826d5999c33dadcd":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_b084661df41c4fa4a69c546ddcaf519c","max":2539,"min":0,"orientation":"horizontal","style":"IPY_MODEL_da2ba9bcf72b44348065ec345f554008","value":2539}},"71f6deb181d04ac495d8be7405a9daea":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3db12e0df160443c8da290f79d903457","placeholder":"​","style":"IPY_MODEL_2db89d52056c47c2b006c44fe89bce32","value":" 2.54k/2.54k [00:00&lt;00:00, 94.1kB/s]"}},"d5d18a69055b496fb29c737c940c6a3d":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2225a24fcae94178a948e2f05d7499db":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d57673cadfd245e49e339f780362d602":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b084661df41c4fa4a69c546ddcaf519c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"da2ba9bcf72b44348065ec345f554008":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"3db12e0df160443c8da290f79d903457":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2db89d52056c47c2b006c44fe89bce32":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b9a1683bd7c44127a700a62df09c2740":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_82755a88b96941869f355139ff33a34c","IPY_MODEL_17dd1a7fa4e84a1481d312dd1e28923a","IPY_MODEL_46a6f706851e4560b89141611e15622a"],"layout":"IPY_MODEL_17e7ff1b5cb7470fac1f0f38121249e3"}},"82755a88b96941869f355139ff33a34c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_220022b208354cee812400428d506b0e","placeholder":"​","style":"IPY_MODEL_cc04a0bcdec04b87a50de06e9608e0a8","value":"Downloading (…)&quot;spiece.model&quot;;: 100%"}},"17dd1a7fa4e84a1481d312dd1e28923a":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_7f4c9a22e5a248dcbe6c4ab8914f2b88","max":791656,"min":0,"orientation":"horizontal","style":"IPY_MODEL_2aca6bfbf4a042bd9cdabac6faea656a","value":791656}},"46a6f706851e4560b89141611e15622a":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_82fa70ea09ad4c4eb2093f98be68e8bd","placeholder":"​","style":"IPY_MODEL_f4f79c7b555743f781455e50e7ad5195","value":" 792k/792k [00:00&lt;00:00, 12.9MB/s]"}},"17e7ff1b5cb7470fac1f0f38121249e3":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"220022b208354cee812400428d506b0e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cc04a0bcdec04b87a50de06e9608e0a8":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7f4c9a22e5a248dcbe6c4ab8914f2b88":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2aca6bfbf4a042bd9cdabac6faea656a":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"82fa70ea09ad4c4eb2093f98be68e8bd":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f4f79c7b555743f781455e50e7ad5195":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"65666ee0fa8a4deba2622e36080ecd97":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_dfc256fdbbde41d383cfd17a816a03af","IPY_MODEL_f0943adb3d7e49acae2884996ef84e55","IPY_MODEL_a10e0d0a45354a8cab4a35b4a386002d"],"layout":"IPY_MODEL_60ad900c6fd34543991b0cc8f0d79a4b"}},"dfc256fdbbde41d383cfd17a816a03af":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_11ff41869104483a931cd5a43fd74c4e","placeholder":"​","style":"IPY_MODEL_a9cfc053290c4a08b22e53c05c42e87b","value":"Downloading (…)/main/tokenizer.json: 100%"}},"f0943adb3d7e49acae2884996ef84e55":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_ed3e08f7e12c42559538eadc46f14809","max":2424064,"min":0,"orientation":"horizontal","style":"IPY_MODEL_6848b004ea514c2c9ae116238dba366a","value":2424064}},"a10e0d0a45354a8cab4a35b4a386002d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3e4929d3364347d78c15da39b2551946","placeholder":"​","style":"IPY_MODEL_83dd446c6cca41d5b223b1aceb2b3a81","value":" 2.42M/2.42M [00:01&lt;00:00, 1.55MB/s]"}},"60ad900c6fd34543991b0cc8f0d79a4b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"11ff41869104483a931cd5a43fd74c4e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a9cfc053290c4a08b22e53c05c42e87b":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ed3e08f7e12c42559538eadc46f14809":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6848b004ea514c2c9ae116238dba366a":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"3e4929d3364347d78c15da39b2551946":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"83dd446c6cca41d5b223b1aceb2b3a81":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ad2b545f2d664bfcb1741a22db968a2c":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_57c9b5f2ac4445678ca22097a0222a03","IPY_MODEL_eb514ffa1ffa4d749f7fbdbb63733f8f","IPY_MODEL_3832a5693a3b40de8b27994db2538349"],"layout":"IPY_MODEL_565869b254044fa38b7df55e68bab5fd"}},"57c9b5f2ac4445678ca22097a0222a03":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_38c015739d2e4c99a3834959c0563950","placeholder":"​","style":"IPY_MODEL_051a111f2e2d4e389cb34475b7d9d5d9","value":"Downloading (…)cial_tokens_map.json: 100%"}},"eb514ffa1ffa4d749f7fbdbb63733f8f":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_56bf48cbe4aa48c3aea421cb37812259","max":2201,"min":0,"orientation":"horizontal","style":"IPY_MODEL_c7990b0e47894b84af6b799ac005e994","value":2201}},"3832a5693a3b40de8b27994db2538349":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5a31c0906b7d4fc38689030779748823","placeholder":"​","style":"IPY_MODEL_9adaf24ddb6b4ac0a8cda15269e5106f","value":" 2.20k/2.20k [00:00&lt;00:00, 58.4kB/s]"}},"565869b254044fa38b7df55e68bab5fd":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"38c015739d2e4c99a3834959c0563950":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"051a111f2e2d4e389cb34475b7d9d5d9":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"56bf48cbe4aa48c3aea421cb37812259":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c7990b0e47894b84af6b799ac005e994":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"5a31c0906b7d4fc38689030779748823":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9adaf24ddb6b4ac0a8cda15269e5106f":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"902d51de5d15490b8e40d51cb9d2a6d7":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_f745f516f9814e7293f677e50eaded2f","IPY_MODEL_7761bcc878504db19a4b2662cfdf61a9","IPY_MODEL_ab1cfad8dc994780b6b980b751531e0e"],"layout":"IPY_MODEL_356fa367dabf4efb9c0f858bcd582db5"}},"f745f516f9814e7293f677e50eaded2f":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a76f15514a734a2ca57c95d6e5d28c1a","placeholder":"​","style":"IPY_MODEL_2895697d2a7b4a4988cb782b45744af0","value":"Downloading (…)lve/main/config.json: 100%"}},"7761bcc878504db19a4b2662cfdf61a9":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_c5ff7a768dc6432cab747b037bc99480","max":662,"min":0,"orientation":"horizontal","style":"IPY_MODEL_233a81cabbbe404e87d98de3eb641fc0","value":662}},"ab1cfad8dc994780b6b980b751531e0e":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d7950561457141e98485648c507beae4","placeholder":"​","style":"IPY_MODEL_e2a49b71a2be4b818d34549552158f21","value":" 662/662 [00:00&lt;00:00, 16.4kB/s]"}},"356fa367dabf4efb9c0f858bcd582db5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a76f15514a734a2ca57c95d6e5d28c1a":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2895697d2a7b4a4988cb782b45744af0":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c5ff7a768dc6432cab747b037bc99480":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"233a81cabbbe404e87d98de3eb641fc0":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"d7950561457141e98485648c507beae4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e2a49b71a2be4b818d34549552158f21":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8b43cd40eb594e3d974fb47a63013eba":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_ca360690cbc348fcb0b594cc56434017","IPY_MODEL_b95287e1033b4645a21029cb4ffab619","IPY_MODEL_81ea2572afde4cb09ac364307785d38d"],"layout":"IPY_MODEL_23bbfb2b32044784a49e7b08b90e1302"}},"ca360690cbc348fcb0b594cc56434017":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_33ca6db178b84da19661fc3d62e09abb","placeholder":"​","style":"IPY_MODEL_2da3ecb995514cb7a2fac91f49165f24","value":"Downloading (…)&quot;pytorch_model.bin&quot;;: 100%"}},"b95287e1033b4645a21029cb4ffab619":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_994a0a3fb8de4798bc5c3e34b27d2aa0","max":3132781861,"min":0,"orientation":"horizontal","style":"IPY_MODEL_15896f0b77164af7bdeb365755b3d478","value":3132781861}},"81ea2572afde4cb09ac364307785d38d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f1566112e762430397c9116cca9aa39b","placeholder":"​","style":"IPY_MODEL_d33d0c19f0b542c5ac07ac1f2858e3a7","value":" 3.13G/3.13G [00:12&lt;00:00, 250MB/s]"}},"23bbfb2b32044784a49e7b08b90e1302":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"33ca6db178b84da19661fc3d62e09abb":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2da3ecb995514cb7a2fac91f49165f24":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"994a0a3fb8de4798bc5c3e34b27d2aa0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"15896f0b77164af7bdeb365755b3d478":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"f1566112e762430397c9116cca9aa39b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d33d0c19f0b542c5ac07ac1f2858e3a7":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b135be3639114a6ab0c587f08a889c82":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_9d8979ec18384dc2a297ce9a68d9bd9c","IPY_MODEL_3cce22420ff046d9872b08ce23e725a1","IPY_MODEL_f057f834fc8c45e4b46809f2ee1abad9"],"layout":"IPY_MODEL_2b0d399693ec4f8b9065e477fb573e1a"}},"9d8979ec18384dc2a297ce9a68d9bd9c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d53e8179d299450083ddf7e2cb1e473e","placeholder":"​","style":"IPY_MODEL_7b38475b1275447cae202d0b60e6a207","value":"Downloading (…)neration_config.json: 100%"}},"3cce22420ff046d9872b08ce23e725a1":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_c27eec62191a46b7b37dbb72d3e7e864","max":147,"min":0,"orientation":"horizontal","style":"IPY_MODEL_5d250db552db494a815fa3ea51592e68","value":147}},"f057f834fc8c45e4b46809f2ee1abad9":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_fd3fec011f284d05b5d7d5e906174582","placeholder":"​","style":"IPY_MODEL_a40e99f09452422b9460d9ec7a5f127e","value":" 147/147 [00:00&lt;00:00, 5.18kB/s]"}},"2b0d399693ec4f8b9065e477fb573e1a":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d53e8179d299450083ddf7e2cb1e473e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7b38475b1275447cae202d0b60e6a207":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c27eec62191a46b7b37dbb72d3e7e864":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5d250db552db494a815fa3ea51592e68":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"fd3fec011f284d05b5d7d5e906174582":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a40e99f09452422b9460d9ec7a5f127e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","source":["# Instalacja pakietów"],"metadata":{"id":"ZXsOR6oJOJbd"}},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"8l0hzptKNiZS","executionInfo":{"status":"ok","timestamp":1676655752895,"user_tz":-60,"elapsed":22903,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"outputId":"e2ee6c4b-82e8-40b0-8c81-9c0b63be98f1"},"outputs":[{"output_type":"stream","name":"stdout","text":["Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n","Collecting transformers\n","  Downloading transformers-4.26.1-py3-none-any.whl (6.3 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.3/6.3 MB\u001b[0m \u001b[31m40.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting datasets\n","  Downloading datasets-2.9.0-py3-none-any.whl (462 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m462.8/462.8 KB\u001b[0m \u001b[31m24.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: torch in /usr/local/lib/python3.8/dist-packages (1.13.1+cu116)\n","Collecting sentencepiece\n","  Downloading sentencepiece-0.1.97-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m29.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers) (3.9.0)\n","Collecting tokenizers!=0.11.3,<0.14,>=0.11.1\n","  Downloading tokenizers-0.13.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.6 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.6/7.6 MB\u001b[0m \u001b[31m59.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from transformers) (2.25.1)\n","Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (2022.6.2)\n","Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (23.0)\n","Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (6.0)\n","Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.8/dist-packages (from transformers) (4.64.1)\n","Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (1.21.6)\n","Collecting huggingface-hub<1.0,>=0.11.0\n","  Downloading huggingface_hub-0.12.1-py3-none-any.whl (190 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.3/190.3 KB\u001b[0m \u001b[31m7.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.8/dist-packages (from datasets) (1.3.5)\n","Requirement already satisfied: aiohttp in /usr/local/lib/python3.8/dist-packages (from datasets) (3.8.4)\n","Requirement already satisfied: dill<0.3.7 in /usr/local/lib/python3.8/dist-packages (from datasets) (0.3.6)\n","Collecting multiprocess\n","  Downloading multiprocess-0.70.14-py38-none-any.whl (132 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m132.0/132.0 KB\u001b[0m \u001b[31m9.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.8/dist-packages (from datasets) (2023.1.0)\n","Requirement already satisfied: pyarrow>=6.0.0 in /usr/local/lib/python3.8/dist-packages (from datasets) (9.0.0)\n","Collecting xxhash\n","  Downloading xxhash-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (213 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m213.0/213.0 KB\u001b[0m \u001b[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting responses<0.19\n","  Downloading responses-0.18.0-py3-none-any.whl (38 kB)\n","Requirement already satisfied: typing-extensions in /usr/local/lib/python3.8/dist-packages (from torch) (4.5.0)\n","Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.3)\n","Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (3.0.1)\n","Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.1)\n","Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (6.0.4)\n","Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (4.0.2)\n","Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.8.2)\n","Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (22.2.0)\n","Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2.10)\n","Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (4.0.0)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2022.12.7)\n","Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (1.24.3)\n","Collecting urllib3<1.27,>=1.21.1\n","  Downloading urllib3-1.26.14-py2.py3-none-any.whl (140 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m140.6/140.6 KB\u001b[0m \u001b[31m9.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas->datasets) (2.8.2)\n","Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas->datasets) (2022.7.1)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil>=2.7.3->pandas->datasets) (1.15.0)\n","Installing collected packages: tokenizers, sentencepiece, xxhash, urllib3, multiprocess, responses, huggingface-hub, transformers, datasets\n","  Attempting uninstall: urllib3\n","    Found existing installation: urllib3 1.24.3\n","    Uninstalling urllib3-1.24.3:\n","      Successfully uninstalled urllib3-1.24.3\n","Successfully installed datasets-2.9.0 huggingface-hub-0.12.1 multiprocess-0.70.14 responses-0.18.0 sentencepiece-0.1.97 tokenizers-0.13.2 transformers-4.26.1 urllib3-1.26.14 xxhash-3.2.0\n"]}],"source":["!pip install transformers datasets torch sentencepiece"]},{"cell_type":"markdown","source":["# Załadowanie datasetu"],"metadata":{"id":"dhN0rmb5Oi3d"}},{"cell_type":"code","source":["from datasets import load_dataset"],"metadata":{"id":"tnaDkwZ2Pbnn","executionInfo":{"status":"ok","timestamp":1676655754295,"user_tz":-60,"elapsed":1405,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}}},"execution_count":2,"outputs":[]},{"cell_type":"code","source":["dataset = load_dataset(\"sms_spam\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":232,"referenced_widgets":["c9c5041c2d734e449a3f480d27c6930f","93c307f83f8345569c6d5eb992facb8f","0b1c4594b3394af9bf6bdb1f5c21efde","8acffba7f8d841bca237676946643b85","6cd1d313107b458fa45e5872c9ce66f0","d6a531cb1e5442398ad46a7555af15ff","a682581443054d6ab11c4445b59ebaf8","7eca2a8a527c417794f7902354dff4fa","44c053a9f38d42ff8b1a37e7892a87d2","086a54cdf54a4631b1d8aa8c835f74a5","5334c04227694b0dbfca313b8b3c1b73","ed0315a7a3734a9096b646d6087113d5","63e57336638c43269033927fea082b5e","5c74805203c149a28b39b8a6673b3226","bb1356f3522c48779c6ef331fd0cb258","a9764585f1304fac85a4647205a07248","fe9a18c0f645464687a47a598c29856c","4cac33b53d5c47bf9e2559a19d786ba7","c9c783ba8f2c47d2a63f6008c6c7ebf6","97a86f253eed42059461528ac274ff5f","db85eaf5afc64c258db557856084caa7","08e5985a87ce49469632b644a0be1e4a","6705f6b0f9754c0ebc4cb0dc1a37ac94","3843c8652968472fb3bfb4ce97b7c45b","a248a43d17ab4633b88a1621942d4757","37c5cf19bd0646aa9ce53218326424f7","f55517fb17e0463e8c8b01f358534914","fddbdaedf5654097942c38002a299e3f","10ed5de382e244d8a56c47c76d90748f","e559dfd597ad454b8d8c2ee005901a35","f48f84cd18ab43b39bf2ca69599ad8c7","1c40bc5715ea448697ac53773d86237a","fc419546e84144349e9c533a8176d4e4","9b3a088c8a7a49b496c6b2e5d2d474dd","a803fe9e84344cdfac171dc6343121e5","12f4b7d3045340d99001a098e0600c64","a92495c6264a4256b9d1c4dd207d9041","c0f1d46ebbdc4bfe8d31985f71875962","e2a740892ddf477ba0b404b1e7e2cd88","b4eb68bbd452409f8faf6f77093f84da","9f8452af363d4a0daf38a7edf7d0a540","ec09103848594bc38e167a16a407cc04","5b8f226314054f1f893d922347c31055","9e4f882e90234dbca0702be76099be77","ef20074dae184d8cb1ead4ce74c2df0b","b4721cb7a4f1486384078edb29a219f5","b8194c781462455ab6b975689d934be3","4cafb17414cf4413b70b04381b3783ca","70819e011c9a4e759758ec8b1e6116f4","bfd5db4a4c9f4a0a9165e3b3a34f6e32","50586a5a007f438d94dad66ad03ca1ba","7295282fb11d4c95aac18ce9047f4da2","1354813182d7427b87f56a8aafc06c34","eaadb5abf84d455994f5378aac2a293c","dc7f94f8535a4734a3cd9fc9cef5fc2e","e5985e951e854e4fbdc8983b57830209","14ecc67173cd4eb2a20d51f817a01a29","bd66d5b4736647a89ec35d65d55f7556","eece11a05e874005ae2532f8139490d0","7ba99dc8284d436f83a341d914e3b226","91b51e4d897843648bc69a4dc684f4bf","def43747b4624cd693383277e8520e1a","438240aff4aa4c30ab5dc8a1159f32be","12772e6a0d1d40e9934ba6c663896c17","68911a228fc343bbb84fd7f6b53f5fb4","9de6fd732fbb4ff5b917f8250710d342"]},"id":"cCiAuRqrOkvV","executionInfo":{"status":"ok","timestamp":1676655766016,"user_tz":-60,"elapsed":11725,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"outputId":"71c3bce0-d144-4064-c952-e39350da5879"},"execution_count":3,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading builder script:   0%|          | 0.00/3.21k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"c9c5041c2d734e449a3f480d27c6930f"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading metadata:   0%|          | 0.00/1.69k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"ed0315a7a3734a9096b646d6087113d5"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading readme:   0%|          | 0.00/4.87k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6705f6b0f9754c0ebc4cb0dc1a37ac94"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Downloading and preparing dataset sms_spam/plain_text to /root/.cache/huggingface/datasets/sms_spam/plain_text/1.0.0/53f051d3b5f62d99d61792c91acefe4f1577ad3e4c216fb0ad39e30b9f20019c...\n"]},{"output_type":"display_data","data":{"text/plain":["Downloading data:   0%|          | 0.00/203k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"9b3a088c8a7a49b496c6b2e5d2d474dd"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Generating train split:   0%|          | 0/5574 [00:00<?, ? examples/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"ef20074dae184d8cb1ead4ce74c2df0b"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Dataset sms_spam downloaded and prepared to /root/.cache/huggingface/datasets/sms_spam/plain_text/1.0.0/53f051d3b5f62d99d61792c91acefe4f1577ad3e4c216fb0ad39e30b9f20019c. Subsequent calls will reuse this data.\n"]},{"output_type":"display_data","data":{"text/plain":["  0%|          | 0/1 [00:00<?, ?it/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"e5985e951e854e4fbdc8983b57830209"}},"metadata":{}}]},{"cell_type":"code","source":["dataset['train'][0]"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"JKFHPko3OnAV","executionInfo":{"status":"ok","timestamp":1676655766016,"user_tz":-60,"elapsed":19,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"outputId":"568dc1d5-85a9-444a-e75a-4632ec63707d"},"execution_count":4,"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\\n',\n"," 'label': 0}"]},"metadata":{},"execution_count":4}]},{"cell_type":"markdown","source":["# Przygotowanie datasetu"],"metadata":{"id":"l140vJrgYxPr"}},{"cell_type":"code","source":["parsed_dataset = []\n","\n","for row in dataset['train']:\n","  text = row['sms'].replace(\"\\n\", \"\")\n","  new_row = {}\n","  new_row['sms'] = text\n","  if row['label'] == 0:\n","    new_row['label'] = \"False\"\n","  else:\n","    new_row['label'] = \"True\"\n","  parsed_dataset.append(new_row)\n","\n","parsed_dataset[0]"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"1boUF-YiY3_y","executionInfo":{"status":"ok","timestamp":1676655766703,"user_tz":-60,"elapsed":703,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"outputId":"1855e54f-cca7-46d4-b456-b7ef96820992"},"execution_count":5,"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...',\n"," 'label': 'False'}"]},"metadata":{},"execution_count":5}]},{"cell_type":"markdown","source":["# Tokenizer FLAN-T5"],"metadata":{"id":"O-J-jBDxPJcn"}},{"cell_type":"code","source":["from transformers import AutoTokenizer"],"metadata":{"id":"P23AYPX1PZ6g","executionInfo":{"status":"ok","timestamp":1676655769374,"user_tz":-60,"elapsed":2673,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}}},"execution_count":6,"outputs":[]},{"cell_type":"code","source":["tokenizer = AutoTokenizer.from_pretrained('google/flan-t5-large')"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":145,"referenced_widgets":["ac64c83f91604136a8605a458cfdb558","a51333fd9ed341119a8517ead52bed95","b975f63617a6419d826d5999c33dadcd","71f6deb181d04ac495d8be7405a9daea","d5d18a69055b496fb29c737c940c6a3d","2225a24fcae94178a948e2f05d7499db","d57673cadfd245e49e339f780362d602","b084661df41c4fa4a69c546ddcaf519c","da2ba9bcf72b44348065ec345f554008","3db12e0df160443c8da290f79d903457","2db89d52056c47c2b006c44fe89bce32","b9a1683bd7c44127a700a62df09c2740","82755a88b96941869f355139ff33a34c","17dd1a7fa4e84a1481d312dd1e28923a","46a6f706851e4560b89141611e15622a","17e7ff1b5cb7470fac1f0f38121249e3","220022b208354cee812400428d506b0e","cc04a0bcdec04b87a50de06e9608e0a8","7f4c9a22e5a248dcbe6c4ab8914f2b88","2aca6bfbf4a042bd9cdabac6faea656a","82fa70ea09ad4c4eb2093f98be68e8bd","f4f79c7b555743f781455e50e7ad5195","65666ee0fa8a4deba2622e36080ecd97","dfc256fdbbde41d383cfd17a816a03af","f0943adb3d7e49acae2884996ef84e55","a10e0d0a45354a8cab4a35b4a386002d","60ad900c6fd34543991b0cc8f0d79a4b","11ff41869104483a931cd5a43fd74c4e","a9cfc053290c4a08b22e53c05c42e87b","ed3e08f7e12c42559538eadc46f14809","6848b004ea514c2c9ae116238dba366a","3e4929d3364347d78c15da39b2551946","83dd446c6cca41d5b223b1aceb2b3a81","ad2b545f2d664bfcb1741a22db968a2c","57c9b5f2ac4445678ca22097a0222a03","eb514ffa1ffa4d749f7fbdbb63733f8f","3832a5693a3b40de8b27994db2538349","565869b254044fa38b7df55e68bab5fd","38c015739d2e4c99a3834959c0563950","051a111f2e2d4e389cb34475b7d9d5d9","56bf48cbe4aa48c3aea421cb37812259","c7990b0e47894b84af6b799ac005e994","5a31c0906b7d4fc38689030779748823","9adaf24ddb6b4ac0a8cda15269e5106f"]},"id":"q5Jz0E_oPMBr","executionInfo":{"status":"ok","timestamp":1676655779499,"user_tz":-60,"elapsed":10128,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"outputId":"ad9b827a-3914-4125-8689-7c002d6cbdda"},"execution_count":7,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading (…)okenizer_config.json:   0%|          | 0.00/2.54k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"ac64c83f91604136a8605a458cfdb558"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)\"spiece.model\";:   0%|          | 0.00/792k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"b9a1683bd7c44127a700a62df09c2740"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)/main/tokenizer.json:   0%|          | 0.00/2.42M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"65666ee0fa8a4deba2622e36080ecd97"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)cial_tokens_map.json:   0%|          | 0.00/2.20k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"ad2b545f2d664bfcb1741a22db968a2c"}},"metadata":{}}]},{"cell_type":"code","source":["sms = parsed_dataset[0]['sms']\n","print('Original: ', sms)\n","print('Tokenized: ', tokenizer.tokenize(sms))\n","print('Token IDs: ', tokenizer.convert_tokens_to_ids(tokenizer.tokenize(sms)))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"dfxJQpoePsvI","executionInfo":{"status":"ok","timestamp":1676655779500,"user_tz":-60,"elapsed":23,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"outputId":"cb35d66e-ee83-47b0-8ae0-80f82f4efd1d"},"execution_count":8,"outputs":[{"output_type":"stream","name":"stdout","text":["Original:  Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\n","Tokenized:  ['▁Go', '▁until', '▁jur', 'ong', '▁point', ',', '▁crazy', '.', '.', '▁Available', '▁only', '▁in', '▁bug', 'is', '▁', 'n', '▁great', '▁world', '▁la', '▁', 'e', '▁buffet', '...', '▁Cine', '▁there', '▁got', '▁', 'a', 'more', '▁wa', 't', '...']\n","Token IDs:  [1263, 552, 10081, 2444, 500, 6, 6139, 5, 5, 8144, 163, 16, 8143, 159, 3, 29, 248, 296, 50, 3, 15, 15385, 233, 17270, 132, 530, 3, 9, 3706, 8036, 17, 233]\n"]}]},{"cell_type":"markdown","source":["# Few shot learning"],"metadata":{"id":"UpluhM8cU5Ir"}},{"cell_type":"code","source":["print(parsed_dataset[0]) #0\n","print(parsed_dataset[123]) #1\n","print(parsed_dataset[2000]) #0\n","print(parsed_dataset[3002]) #1"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"7uNUkixPU85O","executionInfo":{"status":"ok","timestamp":1676655779500,"user_tz":-60,"elapsed":19,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"outputId":"29ccefa4-8bf2-4753-cb89-7b25260dd24d"},"execution_count":9,"outputs":[{"output_type":"stream","name":"stdout","text":["{'sms': 'Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...', 'label': 'False'}\n","{'sms': 'Todays Voda numbers ending 7548 are selected to receive a $350 award. If you have a match please call 08712300220 quoting claim code 4041 standard rates app', 'label': 'True'}\n","{'sms': \"LMAO where's your fish memory when I need it?\", 'label': 'False'}\n","{'sms': 'This message is free. Welcome to the new & improved Sex & Dogging club! To unsubscribe from this service reply STOP. msgs@150p 18+only', 'label': 'True'}\n"]}]},{"cell_type":"code","source":["non_spam_1 = \"The text message: \" + parsed_dataset[0]['sms'] + \"\\nIs this sms spam? Ans: False\\n\\n\"\n","spam_1 = \"The text message: \" + parsed_dataset[123]['sms'] + \"\\nIs this sms spam? Ans: True\\n\\n\"\n","non_spam_2 = \"The text message: \" + parsed_dataset[2000]['sms'] + \"\\nIs this sms spam? Ans: False\\n\\n\"\n","spam_2 = \"The text message: \" + parsed_dataset[3002]['sms'] + \"\\nIs this sms spam? Ans: True\\n\\n\"\n","\n","few_shot_prefix = non_spam_1 + spam_1 + non_spam_2 + spam_2 + \"The text message: \"\n","print(few_shot_prefix)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"lj0issBznZfK","executionInfo":{"status":"ok","timestamp":1676655779500,"user_tz":-60,"elapsed":17,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"outputId":"c4d190c1-49fa-4a55-f019-a342c165e517"},"execution_count":10,"outputs":[{"output_type":"stream","name":"stdout","text":["The text message: Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...\n","Is this sms spam? Ans: False\n","\n","The text message: Todays Voda numbers ending 7548 are selected to receive a $350 award. If you have a match please call 08712300220 quoting claim code 4041 standard rates app\n","Is this sms spam? Ans: True\n","\n","The text message: LMAO where's your fish memory when I need it?\n","Is this sms spam? Ans: False\n","\n","The text message: This message is free. Welcome to the new & improved Sex & Dogging club! To unsubscribe from this service reply STOP. msgs@150p 18+only\n","Is this sms spam? Ans: True\n","\n","The text message: \n"]}]},{"cell_type":"markdown","source":["# Load FLAN-T5 model"],"metadata":{"id":"okTx_ynMV0rH"}},{"cell_type":"code","source":["from transformers import AutoModelForSeq2SeqLM"],"metadata":{"id":"Eu-7Eed8WgN0","executionInfo":{"status":"ok","timestamp":1676655779501,"user_tz":-60,"elapsed":16,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}}},"execution_count":11,"outputs":[]},{"cell_type":"code","source":["model = AutoModelForSeq2SeqLM.from_pretrained('google/flan-t5-large')\n","\n","model.cuda()"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000,"referenced_widgets":["902d51de5d15490b8e40d51cb9d2a6d7","f745f516f9814e7293f677e50eaded2f","7761bcc878504db19a4b2662cfdf61a9","ab1cfad8dc994780b6b980b751531e0e","356fa367dabf4efb9c0f858bcd582db5","a76f15514a734a2ca57c95d6e5d28c1a","2895697d2a7b4a4988cb782b45744af0","c5ff7a768dc6432cab747b037bc99480","233a81cabbbe404e87d98de3eb641fc0","d7950561457141e98485648c507beae4","e2a49b71a2be4b818d34549552158f21","8b43cd40eb594e3d974fb47a63013eba","ca360690cbc348fcb0b594cc56434017","b95287e1033b4645a21029cb4ffab619","81ea2572afde4cb09ac364307785d38d","23bbfb2b32044784a49e7b08b90e1302","33ca6db178b84da19661fc3d62e09abb","2da3ecb995514cb7a2fac91f49165f24","994a0a3fb8de4798bc5c3e34b27d2aa0","15896f0b77164af7bdeb365755b3d478","f1566112e762430397c9116cca9aa39b","d33d0c19f0b542c5ac07ac1f2858e3a7","b135be3639114a6ab0c587f08a889c82","9d8979ec18384dc2a297ce9a68d9bd9c","3cce22420ff046d9872b08ce23e725a1","f057f834fc8c45e4b46809f2ee1abad9","2b0d399693ec4f8b9065e477fb573e1a","d53e8179d299450083ddf7e2cb1e473e","7b38475b1275447cae202d0b60e6a207","c27eec62191a46b7b37dbb72d3e7e864","5d250db552db494a815fa3ea51592e68","fd3fec011f284d05b5d7d5e906174582","a40e99f09452422b9460d9ec7a5f127e"]},"id":"JKv9O8kfV2zZ","executionInfo":{"status":"ok","timestamp":1676655814395,"user_tz":-60,"elapsed":34910,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"outputId":"2f44a817-5096-4dfb-920e-2b92c3cc95d6"},"execution_count":12,"outputs":[{"output_type":"display_data","data":{"text/plain":["Downloading (…)lve/main/config.json:   0%|          | 0.00/662 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"902d51de5d15490b8e40d51cb9d2a6d7"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)\"pytorch_model.bin\";:   0%|          | 0.00/3.13G [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"8b43cd40eb594e3d974fb47a63013eba"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading (…)neration_config.json:   0%|          | 0.00/147 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"b135be3639114a6ab0c587f08a889c82"}},"metadata":{}},{"output_type":"execute_result","data":{"text/plain":["T5ForConditionalGeneration(\n","  (shared): Embedding(32128, 1024)\n","  (encoder): T5Stack(\n","    (embed_tokens): Embedding(32128, 1024)\n","    (block): ModuleList(\n","      (0): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","              (relative_attention_bias): Embedding(32, 16)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (1): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (2): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (3): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (4): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (5): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (6): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (7): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (8): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (9): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (10): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (11): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (12): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (13): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (14): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (15): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (16): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (17): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (18): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (19): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (20): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (21): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (22): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (23): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","    )\n","    (final_layer_norm): T5LayerNorm()\n","    (dropout): Dropout(p=0.1, inplace=False)\n","  )\n","  (decoder): T5Stack(\n","    (embed_tokens): Embedding(32128, 1024)\n","    (block): ModuleList(\n","      (0): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","              (relative_attention_bias): Embedding(32, 16)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (1): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (2): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (3): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (4): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (5): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (6): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (7): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (8): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (9): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (10): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (11): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (12): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (13): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (14): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (15): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (16): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (17): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (18): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (19): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (20): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (21): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (22): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","      (23): T5Block(\n","        (layer): ModuleList(\n","          (0): T5LayerSelfAttention(\n","            (SelfAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (1): T5LayerCrossAttention(\n","            (EncDecAttention): T5Attention(\n","              (q): Linear(in_features=1024, out_features=1024, bias=False)\n","              (k): Linear(in_features=1024, out_features=1024, bias=False)\n","              (v): Linear(in_features=1024, out_features=1024, bias=False)\n","              (o): Linear(in_features=1024, out_features=1024, bias=False)\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","          (2): T5LayerFF(\n","            (DenseReluDense): T5DenseGatedActDense(\n","              (wi_0): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wi_1): Linear(in_features=1024, out_features=2816, bias=False)\n","              (wo): Linear(in_features=2816, out_features=1024, bias=False)\n","              (dropout): Dropout(p=0.1, inplace=False)\n","              (act): NewGELUActivation()\n","            )\n","            (layer_norm): T5LayerNorm()\n","            (dropout): Dropout(p=0.1, inplace=False)\n","          )\n","        )\n","      )\n","    )\n","    (final_layer_norm): T5LayerNorm()\n","    (dropout): Dropout(p=0.1, inplace=False)\n","  )\n","  (lm_head): Linear(in_features=1024, out_features=32128, bias=False)\n",")"]},"metadata":{},"execution_count":12}]},{"cell_type":"markdown","source":["# Helper functions"],"metadata":{"id":"F_SDAwxoawDy"}},{"cell_type":"code","source":["import torch"],"metadata":{"id":"rdWMg_KJZEZH","executionInfo":{"status":"ok","timestamp":1676655814396,"user_tz":-60,"elapsed":17,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}}},"execution_count":13,"outputs":[]},{"cell_type":"code","source":["def calculate_accuracy(preds, target):\n","  results_ok = 0.0\n","  results_false = 0.0\n","\n","  for idx, pred in enumerate(preds):\n","    if pred == target[idx]:\n","      results_ok += 1.0\n","    else:\n","      results_false += 1.0\n","\n","  return results_ok / (results_ok + results_false)"],"metadata":{"id":"FzUi8908ax61","executionInfo":{"status":"ok","timestamp":1676655814396,"user_tz":-60,"elapsed":16,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}}},"execution_count":14,"outputs":[]},{"cell_type":"code","source":["if torch.cuda.is_available():     \n","    device = torch.device(\"cuda\")\n","\n","    print('There are %d GPU(s) available.' % torch.cuda.device_count())\n","    print('We will use the GPU:', torch.cuda.get_device_name(0))\n","\n","else:\n","    print('No GPU available, using the CPU instead.')\n","    device = torch.device(\"cpu\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"86i7iRmtW-6L","executionInfo":{"status":"ok","timestamp":1676655814397,"user_tz":-60,"elapsed":17,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"outputId":"65d2b65e-db13-4e50-eb08-809dc9abe70a"},"execution_count":15,"outputs":[{"output_type":"stream","name":"stdout","text":["There are 1 GPU(s) available.\n","We will use the GPU: Tesla T4\n"]}]},{"cell_type":"markdown","source":["# Predykcja"],"metadata":{"id":"H_YI3bS3VHQE"}},{"cell_type":"code","source":["parsed_dataset = parsed_dataset[1:123] + parsed_dataset[124:2000] + parsed_dataset[2001:3002] + parsed_dataset[3003:]\n","predictions = []\n","expected = []\n","\n","for row in parsed_dataset:\n","  input_text = few_shot_prefix + row['sms'] + \"\\nIs this sms spam? Ans: \"\n","  input_ids = tokenizer(input_text, return_tensors=\"pt\").input_ids.to(device)\n","\n","  generated_ids = model.generate(input_ids, do_sample=True, temperature=0.9, max_length=200)\n","  generated_text = tokenizer.decode(generated_ids[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)\n","  \n","  predictions.append(generated_text)\n","  expected.append(row['label'])\n","\n","acc = calculate_accuracy(predictions, expected)\n","print(acc)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"vunLGuBGVGmh","executionInfo":{"status":"ok","timestamp":1676656989630,"user_tz":-60,"elapsed":783918,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"outputId":"965f7398-80d9-4bb9-98fb-6ea563a9be38"},"execution_count":17,"outputs":[{"output_type":"stream","name":"stdout","text":["0.20786920589292132\n"]}]},{"cell_type":"code","source":["print(\"Sample prediction: {}, expected: {}\".format(predictions[101], expected[101]))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"20WbYZmLaDl7","executionInfo":{"status":"ok","timestamp":1676657092260,"user_tz":-60,"elapsed":499,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"outputId":"1895e96b-c8a4-431f-961b-39debb09b217"},"execution_count":18,"outputs":[{"output_type":"stream","name":"stdout","text":["Sample prediction: Yes, expected: False\n"]}]},{"cell_type":"markdown","source":["# MCC Score"],"metadata":{"id":"dLYc9WXz_B1o"}},{"cell_type":"code","source":["from sklearn.metrics import matthews_corrcoef\n","\n","print('Calculating Matthews Corr. Coef. for each batch...')\n","matthews = matthews_corrcoef(expected, predictions)    \n","print('Total MCC: %.3f' % matthews)"],"metadata":{"id":"hPEPpXXX_DXR","executionInfo":{"status":"ok","timestamp":1676657097378,"user_tz":-60,"elapsed":1384,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"d2545d4a-002f-4e52-e421-1e094fcb6592"},"execution_count":19,"outputs":[{"output_type":"stream","name":"stdout","text":["Calculating Matthews Corr. Coef. for each batch...\n","Total MCC: 0.050\n"]}]},{"cell_type":"markdown","source":["# Save model"],"metadata":{"id":"GPhCp068_Iwq"}},{"cell_type":"code","source":["from google.colab import drive\n","\n","drive.mount('/content/gdrive/', force_remount=True)\n","\n","output_dir = '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_6_FLAN-T5_Model'\n","print(\"Saving model to %s\" % output_dir)\n","\n","model_to_save = model.module if hasattr(model, 'module') else model\n","model_to_save.save_pretrained(output_dir)\n","tokenizer.save_pretrained(output_dir)"],"metadata":{"id":"avafCMoS_KDF","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1676657142469,"user_tz":-60,"elapsed":41097,"user":{"displayName":"Mike4 Studia4","userId":"02075505067789671500"}},"outputId":"9de1a74c-d045-4213-d20c-867328fae84c"},"execution_count":20,"outputs":[{"output_type":"stream","name":"stdout","text":["Mounted at /content/gdrive/\n","Saving model to /content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_6_FLAN-T5_Model\n"]},{"output_type":"execute_result","data":{"text/plain":["('/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_6_FLAN-T5_Model/tokenizer_config.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_6_FLAN-T5_Model/special_tokens_map.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_6_FLAN-T5_Model/spiece.model',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_6_FLAN-T5_Model/added_tokens.json',\n"," '/content/gdrive/My Drive/UAM/Przetwarzanie-tekstu/Ver_6_FLAN-T5_Model/tokenizer.json')"]},"metadata":{},"execution_count":20}]},{"cell_type":"markdown","source":["# Bibliografia\n","- https://huggingface.co/docs/transformers/main/en/model_doc/flan-t5\n","- https://mccormickml.com/2019/07/22/BERT-fine-tuning/#a1-saving--loading-fine-tuned-model\n","- https://huggingface.co/docs/transformers/model_doc/t5#training"],"metadata":{"id":"wHzm2_nDA6i-"}}]}
\ No newline at end of file