s460983/petite-difference-challenge2
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
master
petite-difference-challenge2/Copy_of_1_PyTorch.ipynb
kabi097 2b5764a013 Zadania
2020-05-14 02:29:26 +02:00

5321 lines
148 KiB
Plaintext
Raw Permalink Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "UwCrntUZQFGT"
},
"source": [
"# Warsztaty z uczenia maszynowego A: problemy rozpoznawania mowy\n",
"## Language Model\n",
"###### 2020.04.17\n",
"---\n",
"### Karol Kaczmarek\n",
"#### karol.kaczmarek [at] amu.edu.pl\n",
"#### pokój B2-36 (proszę o kontakt w razie spotkania)\n",
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "jVv3g1MAQFGU"
},
"source": [
"**Zaliczenia**:\n",
"1. Obecności\n",
"1. Zadania z 2-części warsztatów (2020-04-17, 2020-05-15)\n",
"1. Ocena końcowa to średnia ocena z 2-części\n",
"\n",
"**Agenda**:\n",
"1. PyTorch\n",
"1. Language Model\n",
"1. Architektura Transformer\n",
"1. BERT, GPT-2, RoBERTa\n",
"\n",
"**Linki**:\n",
"- [Harmonogram](https://docs.google.com/document/d/1sGe8Hf_AreDgYRcXPgrtLLDFKj1J4f2xQZmNm92ojW0)\n",
"- [Colaboratory Google](htps://colab.research.google.com/notebooks/welcome.ipynb)\n",
"\n",
"**Literatura**:\n",
"- PyTorch documentation, https://pytorch.org/docs/stable/index.html, dostęp 17.04.2020\n",
"- Vishnu Subramanian, Deep Learning with PyTorch, ISBN: 9781788624336\n",
"- Philipp Koehn, Statistical Machine Translation - Draft of Chapter 13: Neural Machine Translation, https://arxiv.org/pdf/1709.07809.pdf, dostęp 17.04.2020"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "Fr2TIS0kQFGW"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "15YUR67qQFGY"
},
"source": [
"![PyTorch](https://github.com/pytorch/pytorch/raw/master/docs/source/_static/img/pytorch-logo-dark.png)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "4SEjjHxfQFGa"
},
"source": [
"[PyTorch](https://pytorch.org/) is a Python package that provides two high-level features:\n",
"- Tensor computation (like NumPy) with strong GPU acceleration\n",
"- Deep neural networks built on a tape-based autograd system\n",
"\n",
"Usually PyTorch is used either as:\n",
"- a replacement for NumPy to use the power of GPUs.\n",
"- a deep learning research platform that provides maximum flexibility and speed."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "z4GF4TrXQFGb"
},
"source": [
"Links:\n",
"- [pytorch.org](https://pytorch.org/)\n",
"- [PyTorch GitHub](https://github.com/pytorch/pytorch)\n",
"- [PyTorch documentation](https://pytorch.org/docs/stable/index.html)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "V_biQB5XQFGc"
},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "eMxljULBQFGg"
},
"outputs": [],
"source": [
"import torch\n",
"import numpy as np"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "LAZSDz6IQFGm",
"outputId": "22623d27-a9ce-476d-cd2e-a99002270980"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([ 1, 5, 300, 9, 123])\n"
]
}
],
"source": [
"x = torch.tensor([1, 5, 300, 9, 123])\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "M3WaVUJAQFGt",
"outputId": "0a5fe904-2a8d-4738-e311-d95ac54b6935"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<class 'numpy.ndarray'> | [ 1 5 300 9 123]\n",
"<class 'torch.Tensor'> | tensor([ 1, 5, 300, 9, 123])\n"
]
}
],
"source": [
"array = np.array([1, 5, 300, 9, 123])\n",
"print(type(array), '|', array)\n",
"\n",
"tensor = torch.tensor(array)\n",
"print(type(tensor), ' |', tensor)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "5z7ALSy7QFGx",
"outputId": "29b65d16-f289-4c23-af66-a01425ba3b58"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[4.3199e-05, 3.1458e-12, 7.9309e+34, 6.0022e+31, 4.2964e+24],\n",
" [7.3162e+28, 1.6992e-07, 6.7331e+22, 6.7120e+22, 2.8795e+32]])\n"
]
}
],
"source": [
"tensor = torch.Tensor(2, 5)\n",
"print(tensor)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "SI_Px8ePQFG3"
},
"source": [
"[torch.empty()](https://pytorch.org/docs/stable/torch.html#torch.empty)\n",
"[torch.zeros()](https://pytorch.org/docs/stable/torch.html#torch.zeros)\n",
"[torch.ones()](https://pytorch.org/docs/stable/torch.html#torch.ones)\n",
"[torch.rand()](https://pytorch.org/docs/stable/torch.html#torch.rand)\n",
"[torch.randn()](https://pytorch.org/docs/stable/torch.html#torch.randn)\n",
"[torch.full()](https://pytorch.org/docs/stable/torch.html#torch.full)\n",
"[torch.arange()](https://pytorch.org/docs/stable/torch.html#torch.arange)\n",
"[torch.linspace()](https://pytorch.org/docs/stable/torch.html#torch.linspace)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "6uBkS1c4QFG4",
"outputId": "df2b7980-87ad-40ba-ca32-d57ceefba547"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([3.3827e+00, 4.5852e-41, 3.3827e+00, 4.5852e-41, 4.4842e-44])\n"
]
}
],
"source": [
"x = torch.empty(5)\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "6VBfCHn6QFG8",
"outputId": "bbde8692-6df0-455d-ee8b-3b29bdc2c71a"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([0., 0., 0., 0., 0.])\n"
]
}
],
"source": [
"x = torch.zeros(5)\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "St_LIrR9QFHA",
"outputId": "597203a2-b994-4aee-d804-ee90ba8e0764"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([1., 1., 1., 1., 1.])\n"
]
}
],
"source": [
"x = torch.ones(5)\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "FPsga0iKQFHF",
"outputId": "f61b899c-71f7-4404-818f-d440cd6c61c3"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([0.3035, 0.8395, 0.3348, 0.1965, 0.0337])\n"
]
}
],
"source": [
"x = torch.rand(5)\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "kSD26LlwQFHJ",
"outputId": "a7c4e935-deae-454a-b254-6ea0b41bbdcc"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([-1.8783, -0.4755, -0.9203, 0.6231, 0.1235])\n"
]
}
],
"source": [
"x = torch.randn(5)\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "2MSGQORcQFHO",
"outputId": "36bf782c-0fb5-4b61-c5d1-6f32370b9b2c"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[5., 5., 5.],\n",
" [5., 5., 5.]])\n"
]
}
],
"source": [
"x = torch.full((2, 3), 5)\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 202
},
"colab_type": "code",
"id": "cG6gtfjMQFHR",
"outputId": "0c327d84-e5ec-444e-fe11-27f3f8a30ec8"
},
"outputs": [],
"source": [
"x = torch.full((2, 3), 5)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "JCHtwYZoQFHW",
"outputId": "4ccaabed-72fb-4f6b-f14c-ec27372b9a4b"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([0, 1, 2, 3, 4])\n",
"tensor([2, 3, 4])\n"
]
}
],
"source": [
"x = torch.arange(5)\n",
"print(x)\n",
"x = torch.arange(2, 5)\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "ydpLAeOQQFHa",
"outputId": "98970673-c360-468c-a292-be2757e6e013"
},
"outputs": [
{
"data": {
"text/plain": [
"tensor([1.0000, 1.5000, 2.0000, 2.5000, 3.0000, 3.5000, 4.0000, 4.5000])"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"torch.arange(1.0, 5.0, 0.5)"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "AzXwi1TBQFHf",
"outputId": "85752484-0726-41b5-ead8-4131f50ec9da"
},
"outputs": [
{
"data": {
"text/plain": [
"tensor([ 3.0000, 4.7500, 6.5000, 8.2500, 10.0000])"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"torch.linspace(3, 10, steps=5)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "5cMSBXM3QFHi",
"outputId": "630fe9a6-dae5-44da-ad39-9c5586038904"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([ 1.0000, 3.2500, 5.5000, 7.7500, 10.0000])\n"
]
}
],
"source": [
"x = torch.linspace(1, 10, steps=5)\n",
"print(x)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "KVZQXEx2QFHm"
},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "RQwEFQUSQFHn",
"outputId": "18d38298-ec21-4371-c73b-c72ca8d9d8c0"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([0.8173, 0.8403, 0.7391, 0.2665, 0.8639])\n"
]
}
],
"source": [
"x = torch.rand(5)\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "0nwJFwZeQFHq",
"outputId": "cb258add-3b48-4a76-bcef-d96f6fdd85ba"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor(0.7391)\n",
"tensor([0.8403, 0.7391])\n"
]
}
],
"source": [
"print(x[2])\n",
"print(x[1:3])"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "JafVDJjyQFHt",
"outputId": "bda496e1-8ba5-4133-9804-5bba756774b6"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([ 0.8173, 0.8403, 99.0000, 0.2665, 0.8639])\n"
]
}
],
"source": [
"x[2] = 99\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "xu0EnCFBQFHx",
"outputId": "d14787e7-cc84-43f5-ff8d-e7cd830be283"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([ 0.8173, 1.0000, 99.0000, 0.2665, 0.8639])\n"
]
}
],
"source": [
"x[1] = True\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 86
},
"colab_type": "code",
"id": "bdDX7V5_QFH0",
"outputId": "2cc83108-91b9-435a-beaa-15755d728ca7"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[0.0321, 0.9135, 0.9144, 0.1919, 0.9376, 0.8586, 0.9644],\n",
" [0.8811, 0.0165, 0.6718, 0.4107, 0.7261, 0.5029, 0.6465],\n",
" [0.5959, 0.2529, 0.4962, 0.5259, 0.6860, 0.7571, 0.8950],\n",
" [0.0176, 0.7755, 0.5761, 0.9369, 0.4213, 0.7168, 0.6567]])\n"
]
}
],
"source": [
"x = torch.rand(4, 7)\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "LyzUdV_MQFH3",
"outputId": "e9d76bdb-f487-438d-cc19-68dc058ddf5c"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([0.9644, 0.6465, 0.8950, 0.6567])\n"
]
}
],
"source": [
"print(x[:,-1])"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "LsGfgYynQFH7",
"outputId": "f9dd2de8-edcc-4fcd-edab-0b0675d6e215"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[0.6718, 0.4107, 0.7261],\n",
" [0.4962, 0.5259, 0.6860]])\n"
]
}
],
"source": [
"print(x[1:3,2:5])"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 86
},
"colab_type": "code",
"id": "hEO522NAQFH_",
"outputId": "5cef6072-2fb7-460c-fbc9-3bdf69c1b4ab"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[1.0000, 2.0000, 0.9144, 0.1919, 0.9376, 0.8586, 0.9644],\n",
" [0.8811, 0.0165, 0.6718, 0.4107, 0.7261, 0.5029, 0.6465],\n",
" [0.5959, 0.2529, 0.4962, 0.5259, 0.6860, 0.7571, 0.8950],\n",
" [0.0176, 0.7755, 0.5761, 0.9369, 0.4213, 0.7168, 0.6567]])\n"
]
}
],
"source": [
"x[0,0:2] = torch.Tensor([1, 2])\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "T8ISF4KrQFIC",
"outputId": "2f6eac9d-412b-431d-a4f6-59cfcba111c8"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[10., 34., 59., 52.],\n",
" [21., 66., 79., 66.]])\n"
]
}
],
"source": [
"z = torch.empty(2, 4).random_(5, 80)\n",
"print(z)"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 156
},
"colab_type": "code",
"id": "WDjWzS24QFIH",
"outputId": "fe145407-5905-4cf7-e152-c57f81cfca51"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[1.0000e+00, 2.0000e+00, 9.1440e-01, 1.9187e-01, 9.3764e-01, 8.5855e-01,\n",
" 9.6439e-01],\n",
" [8.8110e-01, 1.6510e-02, 6.7177e-01, 4.1072e-01, 7.2609e-01, 5.0288e-01,\n",
" 6.4647e-01],\n",
" [5.9593e-01, 1.0000e+01, 3.4000e+01, 5.9000e+01, 5.2000e+01, 7.5711e-01,\n",
" 8.9498e-01],\n",
" [1.7649e-02, 2.1000e+01, 6.6000e+01, 7.9000e+01, 6.6000e+01, 7.1679e-01,\n",
" 6.5671e-01]])\n"
]
}
],
"source": [
"x[2:2 + z.size(0), 1:1 + z.size(1)] = z\n",
"print(x)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "IrzvL5wUQFIL"
},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "VphnkmRrQFIM",
"outputId": "0aab88be-abd8-4524-c25b-0d526c83e010"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([6])\n",
"tensor([0.9270, 0.2327, 0.3381, 0.7915, 0.7357, 0.4224])\n"
]
}
],
"source": [
"x = torch.rand(6)\n",
"print(x.size())\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 139
},
"colab_type": "code",
"id": "pCUmOsWWQFIT",
"outputId": "2ca8312d-137f-4bfa-ce9b-401780930e4d"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([6, 3])\n",
"tensor([[0.4974, 0.4085, 0.0589],\n",
" [0.4546, 0.2406, 0.0320],\n",
" [0.8759, 0.9207, 0.6263],\n",
" [0.8034, 0.9810, 0.6037],\n",
" [0.2395, 0.3363, 0.0798],\n",
" [0.6728, 0.6035, 0.7921]])\n"
]
}
],
"source": [
"x = torch.rand(6, 3)\n",
"print(x.size())\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 330
},
"colab_type": "code",
"id": "GLF7RD19QFIW",
"outputId": "46f4554e-e874-45f2-b5ae-d52e4c49407c"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([3, 5, 4])\n",
"tensor([[[0.2329, 0.9153, 0.1941, 0.8649],\n",
" [0.2225, 0.3155, 0.5143, 0.6199],\n",
" [0.4692, 0.2212, 0.5086, 0.6094],\n",
" [0.3478, 0.0144, 0.3675, 0.8303],\n",
" [0.2158, 0.9979, 0.0900, 0.9084]],\n",
"\n",
" [[0.3173, 0.2841, 0.7448, 0.1124],\n",
" [0.8417, 0.6948, 0.2169, 0.2373],\n",
" [0.0954, 0.6364, 0.0184, 0.7757],\n",
" [0.2341, 0.9157, 0.4323, 0.6121],\n",
" [0.0768, 0.7580, 0.1408, 0.0899]],\n",
"\n",
" [[0.1522, 0.3612, 0.5985, 0.4854],\n",
" [0.1454, 0.3195, 0.0663, 0.4226],\n",
" [0.6063, 0.8836, 0.4289, 0.0821],\n",
" [0.2279, 0.4855, 0.8851, 0.7394],\n",
" [0.3143, 0.1334, 0.3592, 0.6609]]])\n"
]
}
],
"source": [
"x = torch.rand(3, 5, 4)\n",
"print(x.size())\n",
"print(x)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "H6FLRYyGQFIY"
},
"source": [
"[nelement()](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.nelement), [numel()](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.numel)"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "CNf0_723QFIY",
"outputId": "cc5d2ad2-7e41-4b4d-a221-c10f7aa02ee8"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([4, 5])\n",
"20\n"
]
}
],
"source": [
"x = torch.randn(4, 5)\n",
"print(x.size())\n",
"print(x.numel())"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "_Ee7eQqrQFIb"
},
"source": [
"[item()](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.item)"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "rL_HCfWXQFIb",
"outputId": "6f4772f1-6de3-4a21-f0b4-f38ce7a7390d"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([-1.2007])\n",
"-1.200722336769104\n"
]
}
],
"source": [
"x = torch.randn(1)\n",
"print(x)\n",
"print(x.item())"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "ODlkxWFhQFIe",
"outputId": "6349234d-50a2-47c3-de5a-e22d87d59a9b"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([ 0.2868, 0.1182, 1.0450, -1.4448, 2.7314])\n",
"1.0450468063354492\n"
]
}
],
"source": [
"x = torch.randn(5)\n",
"print(x)\n",
"print(x[2].item())"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 167
},
"colab_type": "code",
"id": "sPgT2NueQFIg",
"outputId": "80743df4-3eb1-431b-f1aa-bd4b1dc2c503"
},
"outputs": [
{
"ename": "ValueError",
"evalue": "only one element tensors can be converted to Python scalars",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-36-95a4debbca18>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mValueError\u001b[0m: only one element tensors can be converted to Python scalars"
]
}
],
"source": [
"print(x.item())"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "r1joQUqzQFIj"
},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "FihiwB6FQFIj",
"outputId": "16b7cfa1-be83-4b90-adf9-729681635ed5"
},
"outputs": [],
"source": [
"x = torch.rand(5)\n",
"print(x.type())\n",
"print(x.dtype)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "59SDXkuEQFIm"
},
"source": [
"| Data type | dtype | Tensor types |\n",
"|---|---|---|\n",
"| 32-bit floating point | torch.float32 or torch.float | torch.*.FloatTensor |\n",
"| 64-bit floating point | torch.float64 or torch.double | torch.*.DoubleTensor |\n",
"| 16-bit floating point | torch.float16 or torch.half | torch.*.HalfTensor |\n",
"| 8-bit integer (unsigned) | torch.uint8 | torch.*.ByteTensor |\n",
"| 8-bit integer (signed) | torch.int8 | torch.*.CharTensor |\n",
"| 16-bit integer (signed) | torch.int16 or torch.short | torch.*.ShortTensor |\n",
"| 32-bit integer (signed) | torch.int32 or torch.int | torch.*.IntTensor |\n",
"| 64-bit integer (signed) | torch.int64 or torch.long | torch.*.LongTensor |"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "oDTWlNM7QFIn",
"outputId": "76b8d250-174d-425d-d036-3dce2f426bbd"
},
"outputs": [],
"source": [
"x = torch.LongTensor([1, 2, 3])\n",
"print(x)\n",
"print(x.type(), x.dtype)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "ugn1CjbWQFIp",
"outputId": "95ce0ae5-ca36-40a0-9485-d1040fe39aef"
},
"outputs": [],
"source": [
"x = torch.FloatTensor([1, 2, 3])\n",
"print(x)\n",
"print(x.type(), x.dtype)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "HcrDzgLXQFIs",
"outputId": "9fe63d3e-7043-4c80-acc5-780966465bcc"
},
"outputs": [],
"source": [
"x = torch.HalfTensor([1, 2, 3])\n",
"print(x)\n",
"print(x.type(), x.dtype)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "YKVS1tfpQFIt"
},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 104
},
"colab_type": "code",
"id": "UQZmZ4aLQFIu",
"outputId": "028fb054-076f-49f8-ee1f-f349d2f3c8f0"
},
"outputs": [],
"source": [
"x = torch.rand(5)\n",
"print(x)\n",
"print('*' * 25)\n",
"print(x + 5)\n",
"print('*' * 25)\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 139
},
"colab_type": "code",
"id": "PedalUIIQFIw",
"outputId": "17641249-7c80-489b-dc3c-dff88e36eae3"
},
"outputs": [],
"source": [
"x = torch.rand(5)\n",
"y = torch.rand(5)\n",
"print(x)\n",
"print(y)\n",
"print('*' * 48)\n",
"print(x + y)\n",
"print('*' * 48)\n",
"print(x)\n",
"print(y)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "vznedbf7QFIy",
"outputId": "373fb296-22e7-4756-ed28-fb8d6f9636d1"
},
"outputs": [],
"source": [
"x = torch.rand(5)\n",
"print(x)\n",
"print(x.sum())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 121
},
"colab_type": "code",
"id": "mn3oHX32QFIz",
"outputId": "45223159-ae47-4979-e0c9-1bcb0e50a7fb"
},
"outputs": [],
"source": [
"x = torch.rand(5, 4)\n",
"print(x)\n",
"print(x.sum(dim=1))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 121
},
"colab_type": "code",
"id": "hcKxiRmoQFI2",
"outputId": "fdb35c82-01f0-4af3-8276-26865b6d7db7"
},
"outputs": [],
"source": [
"x = torch.rand(5, 4)\n",
"print(x)\n",
"print(x.mean())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 139
},
"colab_type": "code",
"id": "acYv6qc8QFI5",
"outputId": "0b998b59-b194-44a5-f20e-6aa6be2db694"
},
"outputs": [],
"source": [
"x = torch.rand(5, 4)\n",
"print(x)\n",
"print(x.mean(dim=0))\n",
"print(x.mean(dim=1))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 156
},
"colab_type": "code",
"id": "csEARaSWQFI7",
"outputId": "2309c634-54b6-43b0-af9c-316576370a15"
},
"outputs": [],
"source": [
"x = torch.rand(5)\n",
"y = torch.rand(5)\n",
"print(x)\n",
"print(y)\n",
"print('*' * 48)\n",
"print(x.add(y))\n",
"print(torch.add(x, y))\n",
"print('*' * 48)\n",
"print(x)\n",
"print(y)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 139
},
"colab_type": "code",
"id": "Ior1_cb6QFI-",
"outputId": "091e7bac-5c70-49ac-da62-173fbf669fc2"
},
"outputs": [],
"source": [
"x = torch.rand(5)\n",
"y = torch.rand(5)\n",
"print(x)\n",
"print(y)\n",
"print('*' * 48)\n",
"print(x.add_(y))\n",
"print('*' * 48)\n",
"print(x)\n",
"print(y)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "TqTYz_DxQFJB"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "7AkZE5T_QFJB"
},
"source": [
"[eq()](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.eq) [equal()](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.equal) [lt()](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.lt) [le()](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.le) [gt()](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.gt) [ge()](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.ge)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 86
},
"colab_type": "code",
"id": "s9zBNCNvQFJB",
"outputId": "9fc3f788-1d85-4f35-be4f-668abe74c4e8"
},
"outputs": [],
"source": [
"x = torch.randn(2, 3)\n",
"y = torch.randn(2, 3)\n",
"y[0][1] = x[0][1]\n",
"print(x)\n",
"print(y)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 69
},
"colab_type": "code",
"id": "YWNSX_ONQFJD",
"outputId": "941c76c2-4dd7-4f5a-96bc-b26b548efaa9"
},
"outputs": [],
"source": [
"print(torch.equal(x, y))\n",
"print(torch.eq(x, y))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "qkpOPxD8QFJG",
"outputId": "49f95252-ceb1-48d6-8fe9-b245582fb5f4"
},
"outputs": [],
"source": [
"print(torch.gt(x, y))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "nbbnupNPQFJJ",
"outputId": "d37562c5-3427-4546-ab15-daa7ab5c766c"
},
"outputs": [],
"source": [
"print(torch.le(x, y))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "I-sifEY4QFJL"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "23u8-fvbQFJL"
},
"source": [
"[numpy()](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.numpy)\n",
"[torch.from_numpy()](https://pytorch.org/docs/stable/torch.html#torch.from_numpy)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "HrdTKjCRQFJM",
"outputId": "b467229b-ed05-4de2-9817-99b691d740f4"
},
"outputs": [],
"source": [
"x = torch.rand(4)\n",
"y = x.numpy()\n",
"print(type(x), x)\n",
"print(type(y), y)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "tvONs2eVQFJO",
"outputId": "64627073-da87-4da9-e5f9-d73d4618b9e8",
"scrolled": true
},
"outputs": [],
"source": [
"x = np.random.rand(5)\n",
"y = torch.from_numpy(x)\n",
"print(type(x), x)\n",
"print(type(y), y)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "GWS_aWtVQFJQ"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "x4Ls7vQsQFJR"
},
"source": [
"[view()](https://pytorch.org/docs/stable/tensors.html#torch.Tensor.view)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 208
},
"colab_type": "code",
"id": "cul4Z8JbQFJR",
"outputId": "d6e50bfe-7607-4b28-c02c-c27524f378a0"
},
"outputs": [],
"source": [
"x = torch.randn(4, 4)\n",
"y = x.view(16)\n",
"z = x.view(-1, 8)\n",
"print(x.size())\n",
"print(x)\n",
"print(y.size())\n",
"print(y)\n",
"print(z.size())\n",
"print(z)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "NJ3orAqYQFJU"
},
"source": [
"[torch.cat()](https://pytorch.org/docs/stable/torch.html#torch.cat)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 191
},
"colab_type": "code",
"id": "xto8LW-KQFJV",
"outputId": "cc00a0c4-7e30-4bac-fcae-314be56df1b9"
},
"outputs": [],
"source": [
"x = torch.randn(2, 3)\n",
"y = torch.randn(3, 3)\n",
"print(x)\n",
"print(y)\n",
"z = torch.cat((x, y))\n",
"print(z)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 191
},
"colab_type": "code",
"id": "eD0OIy5rQFJX",
"outputId": "a62543be-63dc-4b31-99ad-901762f5bf71"
},
"outputs": [],
"source": [
"x = torch.randn(3, 2)\n",
"y = torch.randn(2, 2)\n",
"print(x)\n",
"print(y)\n",
"z = torch.cat((x, y))\n",
"print(z)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "l46-ZhKNQFJZ"
},
"source": [
"[torch.narrow()](https://pytorch.org/docs/stable/torch.html#torch.narrow)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 191
},
"colab_type": "code",
"id": "LOQxqaejQFJZ",
"outputId": "1afa2c86-165c-483b-d2e9-fd432566630b"
},
"outputs": [],
"source": [
"x = torch.randn(4, 5)\n",
"print(x)\n",
"y = torch.narrow(x, 0, 1, 2)\n",
"print(y)\n",
"z = torch.narrow(x, 1, 0, 3)\n",
"print(z)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "85NWBwr7QFJb"
},
"source": [
"[torch.reshape()](https://pytorch.org/docs/stable/torch.html#torch.reshape)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 260
},
"colab_type": "code",
"id": "4u1lRlJUQFJb",
"outputId": "dbd5b870-2f16-4934-84fb-b9b3bc441cf9"
},
"outputs": [],
"source": [
"x = torch.randn(4, 5)\n",
"print(x)\n",
"y = torch.reshape(x, (2, 2, -1))\n",
"print(y)\n",
"z = torch.reshape(x, (-1, 4))\n",
"print(z)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "KOJBKPEnQFJe"
},
"source": [
"[torch.chunk()](https://pytorch.org/docs/stable/torch.html#torch.chunk)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 121
},
"colab_type": "code",
"id": "1PmQPKY5QFJe",
"outputId": "69b63c8f-3970-484f-cdbd-ab0de478191e"
},
"outputs": [],
"source": [
"x = torch.rand(25)\n",
"print(x.size())\n",
"x = torch.chunk(x, 5)\n",
"for i in x:\n",
" print(i.size())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 121
},
"colab_type": "code",
"id": "nogN0OCqQFJg",
"outputId": "854b3d5c-5da9-411b-aff5-4c950a8a6c3b"
},
"outputs": [],
"source": [
"x = torch.rand(25, 100)\n",
"print(x.size())\n",
"x = torch.chunk(x, 5)\n",
"for i in x:\n",
" print(i.size())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 86
},
"colab_type": "code",
"id": "gz3UwsLKQFJi",
"outputId": "ae768647-ae29-4b87-901b-18400f3670cd"
},
"outputs": [],
"source": [
"x = torch.rand(25, 100)\n",
"print(x.size())\n",
"x = torch.chunk(x, 3, dim=1)\n",
"for i in x:\n",
" print(i.size())"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "cG-ky53nQFJk"
},
"source": [
"[torch.stack()](https://pytorch.org/docs/stable/torch.html#torch.stack)\n",
"[torch.unbind()](https://pytorch.org/docs/stable/torch.html#torch.unbind)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 278
},
"colab_type": "code",
"id": "2LFcl9-4QFJk",
"outputId": "746cedcd-f33e-49d6-b6dd-2390333a3fa9"
},
"outputs": [],
"source": [
"x1 = torch.rand(3, 5)\n",
"x2 = torch.rand(3, 5)\n",
"print(x1.size(), x2.size())\n",
"print(x1)\n",
"print(x2)\n",
"x3 = torch.stack([x1, x2])\n",
"print(x3.size())\n",
"print(x3)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 156
},
"colab_type": "code",
"id": "nmVK82lfQFJm",
"outputId": "cd31317a-1779-47fe-829a-18bd1a1cb879"
},
"outputs": [],
"source": [
"x4, x5 = torch.unbind(x3)\n",
"print(x4.size(), torch.equal(x1, x4))\n",
"print(x4)\n",
"print(x5.size(), torch.equal(x2, x5))\n",
"print(x5)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "pXpV-SWRQFJo"
},
"source": [
"[torch.squeeze()](https://pytorch.org/docs/stable/torch.html#torch.squeeze)\n",
"[torch.unsqueeze()](https://pytorch.org/docs/stable/torch.html#torch.unsqueeze)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 139
},
"colab_type": "code",
"id": "bKuraednQFJo",
"outputId": "42ebdb0b-a68b-4270-d587-500e581dfcda"
},
"outputs": [],
"source": [
"x1 = torch.randn(1, 2, 1, 3)\n",
"print(x1.size())\n",
"print(x1)\n",
"x2 = torch.squeeze(x1)\n",
"print(x2.size())\n",
"print(x2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 156
},
"colab_type": "code",
"id": "qTyiO1xeQFJq",
"outputId": "9d4541b7-5d10-43c0-a5c5-03ccf8b84acc"
},
"outputs": [],
"source": [
"x3 = torch.unsqueeze(x2, dim=1)\n",
"print(x3.size())\n",
"print(x3)\n",
"x4 = torch.unsqueeze(x3, dim=0)\n",
"print(x4.size(), torch.equal(x4, x1))\n",
"print(x4)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "vrk0fqQeQFJt"
},
"source": [
"[torch.t()](https://pytorch.org/docs/stable/torch.html#torch.t) [torch.transpose()](https://pytorch.org/docs/stable/torch.html#torch.transpose)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 104
},
"colab_type": "code",
"id": "ATD1Cur8QFJt",
"outputId": "4dae3e40-6298-45a7-fe88-6140d04518ef"
},
"outputs": [],
"source": [
"x = torch.randn(2, 3)\n",
"print(x)\n",
"print(x.t())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 139
},
"colab_type": "code",
"id": "VPmdf8OYQFJv",
"outputId": "75a4b8f3-a08d-4723-f8a7-3f88bec33a3a"
},
"outputs": [],
"source": [
"print(x)\n",
"x.t_()\n",
"print(x)\n",
"x = x.t()\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 104
},
"colab_type": "code",
"id": "NOlYnda7QFJy",
"outputId": "35beda09-1f9a-4bc9-dbf8-d2909523bb5f"
},
"outputs": [],
"source": [
"x = torch.randn(2, 3)\n",
"print(x)\n",
"print(x.transpose(0, 1))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "k9ZZu_3oQFJ3"
},
"source": [
"[torch.argsort()](https://pytorch.org/docs/stable/torch.html#torch.argsort)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 173
},
"colab_type": "code",
"id": "IppcDCVRQFJ3",
"outputId": "75e0888b-1bf6-4d2e-b0be-3685fce0ae5f"
},
"outputs": [],
"source": [
"a = torch.randn(3, 4)\n",
"print(a)\n",
"print(torch.argsort(a, dim=1))\n",
"print(torch.argsort(a, dim=0))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "9GrO5IPaQFJ4",
"outputId": "b0285912-20fd-47e2-e87a-2a74df4d54a4"
},
"outputs": [],
"source": [
"b = a.sum(dim=1)\n",
"print(b)\n",
"print(torch.argsort(b, dim=0))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "479XExz5QFJ6"
},
"source": [
"[torch.topk()](https://pytorch.org/docs/stable/torch.html#torch.topk)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 121
},
"colab_type": "code",
"id": "6FRQVdfHQFJ7",
"outputId": "fc813c2d-7f59-48be-a674-1c655fd098ba"
},
"outputs": [],
"source": [
"x = torch.rand(10, 25).sum(dim=1)\n",
"print(x)\n",
"print(x.size())\n",
"print(torch.topk(x, 5))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "KR_PITSHQFJ9"
},
"source": [
"[torch.triu()](https://pytorch.org/docs/stable/torch.html#torch.triu)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 191
},
"colab_type": "code",
"id": "xrUmMLlvQFJ9",
"outputId": "44672a7a-0942-4918-e5b3-f8d0c2aa2faf"
},
"outputs": [],
"source": [
"x = torch.rand(5, 4)\n",
"print(x)\n",
"print(torch.triu(x))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 191
},
"colab_type": "code",
"id": "IkZCM91aQFKA",
"outputId": "b9ff9a4a-a254-4bc0-94ee-2878d625a9aa"
},
"outputs": [],
"source": [
"print(torch.triu(x, diagonal=2))\n",
"print(torch.triu(x, diagonal=-1))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "sC_cbKguQFKC"
},
"source": [
"---\n",
"### CUDA"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "smZaKFc-QFKC",
"outputId": "9729eb89-148d-475e-9616-f77f15cd8e86"
},
"outputs": [],
"source": [
"torch.cuda.is_available()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 295
},
"colab_type": "code",
"id": "A7YrlgJ7QFKE",
"outputId": "51410e97-c79e-42ae-b7c3-bd14cb3309bc"
},
"outputs": [],
"source": [
"!nvidia-smi"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "GaYAnWj3QFKF",
"outputId": "3764fe7f-2949-4793-f51f-9c7a2c65edcd"
},
"outputs": [],
"source": [
"torch.cuda.device_count()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "qwf9wHV5QFKH",
"outputId": "98614c1b-f3eb-42e0-c80d-09a8d0c9f846"
},
"outputs": [],
"source": [
"torch.cuda.current_device()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "1h4pZ8R2QFKK",
"outputId": "8036c397-4a12-4003-ca31-b50e4fecf9b2"
},
"outputs": [],
"source": [
"torch.cuda.get_device_name()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "jY6Obt7-QFKN"
},
"outputs": [],
"source": [
"torch.cuda.set_device(0)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "sNIpmpphQFKP"
},
"outputs": [],
"source": [
"torch.cuda.manual_seed(123)\n",
"torch.cuda.manual_seed_all(123)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "BAzpCZl1QFKb",
"outputId": "ed2b1823-5dfa-4d6b-dae9-88485d0ff11e"
},
"outputs": [],
"source": [
"x = torch.rand(15000, 15000)\n",
"print(x.device)\n",
"print(x.size())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "FjMgKS1sQFKc",
"outputId": "b2f9644d-d6e0-47cd-943b-11f76c181806"
},
"outputs": [],
"source": [
"x = x.to('cuda:0')\n",
"print(x.device)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "QlOJW87MQFKd",
"outputId": "c9cf9c4b-f7e6-49cf-a2f8-6f63118f243a"
},
"outputs": [],
"source": [
"x = torch.rand(15000, 15000)\n",
"x = x.cuda()\n",
"print(x.device)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "9HwwYUpXQFKf",
"outputId": "08f1936a-cf81-4147-90c2-886360d6ef9e"
},
"outputs": [],
"source": [
"y = torch.rand(150, 150, device='cuda:0')\n",
"print(y.device)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "xBX8SJAwQFKg",
"outputId": "1ef40c96-33c5-4d44-f9db-455e88fe01bf"
},
"outputs": [],
"source": [
"y = torch.rand(150, 150, device=x.device)\n",
"print(y.device)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "meI0sivGQFKi"
},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "lK8OuYHgQFKi"
},
"outputs": [],
"source": [
"from tqdm.notebook import tqdm"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 86
},
"colab_type": "code",
"id": "_xMMjXHeQFKk",
"outputId": "33fd73e9-e947-4e41-ab1d-15794d60f0b1"
},
"outputs": [],
"source": [
"x = torch.rand(15000, 15000)\n",
"y = x.to('cuda:0')\n",
"print(x.device, x.size())\n",
"print(y.device, y.size())\n",
"print(x[5:10,1])\n",
"print(y[5:10,1])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66,
"referenced_widgets": [
"02b9468a1d214bc08a7f2c3b0fb0be62",
"e3daf6ee9be947c385914a74cb51bac9",
"217de6953d134b028643fe7278574ec0",
"30b59bbc8c9c46bdb03207ea0270e086",
"af84749aacfb4d0bb42fa23df2c9ae5d",
"ab137a3ff493410cb8368d20018a35b7",
"df2736cc330046ba9d461c3d8c1fc00e",
"865a4c3e5c744e928835f5ae5bfbc760"
]
},
"colab_type": "code",
"id": "EBZC0j7LQFKm",
"outputId": "9c3e52b7-831d-4c6d-c9b1-bab2fb43cccf"
},
"outputs": [],
"source": [
"for i in tqdm(range(1500)):\n",
" x += 1"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66,
"referenced_widgets": [
"822929214e8c413bb39934dea80d6f11",
"a9d61d1b1dfb49d08996d1ef19d4d151",
"ddd431b7923647d3bcc53c90c56a2337",
"4e0fff7fd7f74c5e8e7f3b7b74ce1daf",
"768ed45dc77245619fc1a71d370e4170",
"4f76335fb6bf470282b1e8c2d9138fac",
"9cb253edac994418a9f463965b82d83d",
"8aeaa320d7b1411c89c2856c6d447413"
]
},
"colab_type": "code",
"id": "7AK8pPOVQFKs",
"outputId": "00591b61-6f81-4d30-8dd1-af6b68aee9b4"
},
"outputs": [],
"source": [
"for i in tqdm(range(1500)):\n",
" y += 1"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66,
"referenced_widgets": [
"2029f5210be64624bfda9b8dbd4b623c",
"85f2503489d24c2297c3589a806c89c1",
"dfd11ed9a4bc469ca95a22c0fcad7d05",
"9885433b7d4f407582abc288b9dabfdf",
"0b4166703dc6439bb2534e23e3ab9921",
"a863c0ca60ec48c5a4cdd57bc91c0e84",
"16d85c7b33f1437a965dbab57d3d5c79",
"fd8bebdc35054ca285a2ecd0e6a212a5"
]
},
"colab_type": "code",
"id": "MqgRIxU3QFKt",
"outputId": "c9de733a-ecaa-4481-e148-1890fede7136"
},
"outputs": [],
"source": [
"for i in tqdm(range(1500)):\n",
" x.pow(2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66,
"referenced_widgets": [
"be5c0e141898481fabcbca8a251de690",
"206a29154ce846568cc36f000ba7f3cf",
"7b738450b7d3493489274ca67832372d",
"fda4fe65884c40a59d32dd8b97fadb74",
"878a49228de84b2a887c5594de48de29",
"a4aa76d02ea9402aad6299d758cbf58a",
"da9acea59c634fd7b92fbac5c5ccef17",
"d10b8192bfda40b790a8e547ab4c0ea7"
]
},
"colab_type": "code",
"id": "FZZgmEvMQFKw",
"outputId": "1e868f1a-00d5-43ff-a023-e0f374b28e48"
},
"outputs": [],
"source": [
"for i in tqdm(range(1500)):\n",
" y.pow(2)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "9Gm_53h1QFKw"
},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "SuvGZfUtQFKx",
"outputId": "ec805bd9-bdb3-4b04-d0c7-2be8465cd2a1"
},
"outputs": [],
"source": [
"import pickle\n",
"\n",
"vocab = {'This': 0, 'is': 1, 'sentence': 2, 'a': 3}\n",
"print(type(vocab), vocab)\n",
"\n",
"with open('data.pkl', 'wb') as f:\n",
" pickle.dump(vocab, f)\n",
"\n",
"vocab = {}\n",
"print(type(vocab), vocab)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "30PVj_5HQFKz",
"outputId": "20316d77-7314-4ac1-cd53-ec7870cc8e6a"
},
"outputs": [],
"source": [
"!ls data.pkl"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "VZyz3XITQFK1",
"outputId": "412a4d58-5acc-4a24-84b0-fb02dea9284c"
},
"outputs": [],
"source": [
"with open('data.pkl', 'rb') as f:\n",
" vocab = pickle.load(f)\n",
"print(type(vocab), vocab)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "KrdRxtlCQFK2"
},
"source": [
"---\n",
"### Autograd mechanics\n",
"\n",
"![Link](https://raw.githubusercontent.com/pytorch/pytorch/master/docs/source/_static/img/dynamic_graph.gif)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "PK1K0vFaQFK2",
"outputId": "ffc52cdd-17ee-4469-98a7-ff44da2db76c"
},
"outputs": [],
"source": [
"x = torch.ones(2, 2, requires_grad=True)\n",
"print(x)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 69
},
"colab_type": "code",
"id": "3znkNlzdQFK3",
"outputId": "5bdb8b47-3137-4e41-d7bf-c4cdb9816dcd"
},
"outputs": [],
"source": [
"y = x + 2\n",
"print(y)\n",
"print(y.grad_fn)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 104
},
"colab_type": "code",
"id": "bdp71JBBQFK5",
"outputId": "9be9b767-ce72-4518-d73d-9dc18a5abbae"
},
"outputs": [],
"source": [
"z = y * y * 3\n",
"out = z.mean()\n",
"print(z, out)\n",
"\n",
"print(x.grad)\n",
"out.backward()\n",
"print(x.grad)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "95tNFoC2QFK7",
"outputId": "141147a2-60f7-4252-bead-974d10fcbe4f"
},
"outputs": [],
"source": [
"a = torch.randn(2, 2)\n",
"a *= 2\n",
"print(a.requires_grad)\n",
"a.requires_grad_(True)\n",
"print(a.requires_grad)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 69
},
"colab_type": "code",
"id": "FioUqRVsQFK8",
"outputId": "0bb5eb4d-7789-4a8e-c290-6e5a2dce70bb"
},
"outputs": [],
"source": [
"x = torch.randn(3, requires_grad=True)\n",
"print(x.requires_grad)\n",
"print((x ** 2).requires_grad)\n",
"\n",
"with torch.no_grad():\n",
" print((x ** 2).requires_grad)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "GSGSoTygQFK9"
},
"source": [
"### Neural Network\n",
"\n",
"[torch.nn](https://pytorch.org/docs/stable/nn.html)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "vbYcwnCEQFK9"
},
"source": [
"### Linear layer\n",
"\n",
"$$ Y = Wx + b $$\n",
"\n",
"![add image](https://i.imgur.com/NCdsAPp.png)\n",
"- dense or fully connected layers\n",
"- [torch.nn.Linear](https://pytorch.org/docs/stable/nn.html#torch.nn.Linear)\n",
"- `torch.nn.Linear(in_features=10, out_features=5, bias=True)`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "xwySkSh-QFK-",
"outputId": "5c7012d4-aa76-43c6-acf7-1b0660f75723"
},
"outputs": [],
"source": [
"m = torch.nn.Linear(100, 200)\n",
"print(m)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "mdXDMVQ7QFK_",
"outputId": "25c117ef-1101-42e9-a571-72ea35b885c5"
},
"outputs": [],
"source": [
"x = torch.rand(25, 100)\n",
"y = m(x)\n",
"print(x.size(), '->', y.size())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "cdpLY1zyQFLB",
"outputId": "692141f5-6183-4577-e128-b4e7156b1e0a"
},
"outputs": [],
"source": [
"x = torch.rand(10, 3, 25, 100)\n",
"y = m(x)\n",
"print(x.size(), '->', y.size())"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "ucAz2tODQFLC"
},
"source": [
"#### Non-linear activations\n",
"\n",
"#### Sigmoid\n",
"$$ \\sigma (x) = \\frac{\\mathrm{1} }{\\mathrm{1} + e^{-x}} $$\n",
"- [torch.nn.Sigmoid](https://pytorch.org/docs/stable/nn.html#torch.nn.Sigmoid)\n",
"- [torch.nn.functional.sigmoid()](https://pytorch.org/docs/stable/nn.functional.html#torch.nn.functional.sigmoid)\n",
"![Add image!](https://pytorch.org/docs/stable/_images/Sigmoid.png)\n",
"\n",
"#### Tanh\n",
"$$ \\tanh(x) = \\frac{\\sin(x)}{cosh(x)} = \\frac{e^x - e^{-x}}{e^x + e^{-x}} $$\n",
"- [torch.nn.Tanh](https://pytorch.org/docs/stable/nn.html#torch.nn.Tanh)\n",
"- [torch.nn.functional.tanh()](https://pytorch.org/docs/stable/nn.functional.html#torch.nn.functional.tanh)\n",
"![add image](https://pytorch.org/docs/stable/_images/Tanh.png)\n",
"\n",
"#### ReLU\n",
"$$ f(x) = max(0, x) $$\n",
"- [torch.nn.ReLU](https://pytorch.org/docs/stable/nn.html#torch.nn.ReLU)\n",
"- [torch.nn.functional.relu()](https://pytorch.org/docs/stable/nn.functional.html#torch.nn.functional.relu)\n",
"![add image](https://pytorch.org/docs/stable/_images/ReLU.png)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "PhCBXYVjQFLD",
"outputId": "41c7827f-e379-4a91-8a32-e45f760d34cc"
},
"outputs": [],
"source": [
"m = torch.nn.ReLU()\n",
"print(m)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 191
},
"colab_type": "code",
"id": "dqAPvniTQFLF",
"outputId": "be665f38-6623-4497-cfc9-71623bfc1e1f"
},
"outputs": [],
"source": [
"x = torch.rand(5, 6) - 0.5\n",
"print(x)\n",
"print(m(x))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 191
},
"colab_type": "code",
"id": "UiScSmS-QFLG",
"outputId": "2c1335ae-b55b-4c7d-8740-58d63ca3c3ef"
},
"outputs": [],
"source": [
"x = torch.rand(5, 6) - 0.5\n",
"print(x)\n",
"print(torch.nn.functional.relu(x))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "c5OMEYrqQFLK"
},
"source": [
"### One-hot encoding\n",
"- token is represented by a vector of length `N`, where `N` is the size of the vocabulary\n",
"- vocabulary is the total number of unique words in the document\n",
"\n",
"`This is a sentence`\n",
"\n",
"| token | encoding |\n",
"| --- | --- |\n",
"| `This` | 1000 |\n",
"| `is` | 0100 |\n",
"| `a` | 0010 |\n",
"| `sentence` | 0001 |"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "5GLwPTepQFLK"
},
"source": [
"### Word embedding\n",
"- provides a dense representation of aword filled with floating numbers\n",
"- common to use a word embedding of dimension size 50, 100, 256, 300, and sometimes 1,000"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "KB0QDtcEQFLL",
"outputId": "47910fa4-d860-4a2f-854c-d23a2332cdac"
},
"outputs": [],
"source": [
"m = torch.nn.Embedding(10, 3)\n",
"print(m)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 521
},
"colab_type": "code",
"id": "y1KuZmlrQFLM",
"outputId": "2841201e-3f40-467f-afc5-03f791f17f0e"
},
"outputs": [],
"source": [
"x = torch.LongTensor(5, 4).random_(0, 10)\n",
"print(x)\n",
"y = m(x)\n",
"print(y)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 121
},
"colab_type": "code",
"id": "SbdP79bvQFLO",
"outputId": "a7aa89f5-e094-4c52-9f85-c77259ed39a5"
},
"outputs": [],
"source": [
"word_to_idx = {'This': 0, 'is': 1, 'a': 2, 'sentence': 3}\n",
"\n",
"embeds = torch.nn.Embedding(len(word_to_idx), 8)\n",
"\n",
"lookup_tensor = torch.tensor([word_to_idx['This'], word_to_idx['is'], word_to_idx['a']], dtype=torch.long)\n",
"hello_embed = embeds(lookup_tensor)\n",
"\n",
"print(hello_embed)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "-b_yllS6QFLQ"
},
"source": [
"### One-hot encoding vs Word embedding\n",
"Represent a vocabulary of size 20000:\n",
"- one-hot encoding = 20000 x 20000\n",
"- word embedding = 20000 x dimension size"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "pWE2j2WLQFLQ"
},
"source": [
"### Softmax\n",
"- $$ Softmax(x_i) = \\frac{\\exp(x_i)}{\\sum_{j} {\\exp(x_j)}} $$\n",
"- normalized exponential function\n",
"- activation function that turns numbers (logits) into probabilities that sum to one\n",
"- [torch.nn.Softmax](https://pytorch.org/docs/stable/nn.html?highlight=softmax#torch.nn.Softmax) [torch.nn.functional.softmax()](https://pytorch.org/docs/stable/nn.functional.html?highlight=softmax#torch.nn.functional.softmax)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "4mEES_YRQFLQ",
"outputId": "bc51ccce-191e-4b48-b1c3-5409b4146f20"
},
"outputs": [],
"source": [
"m = torch.nn.Softmax(dim=1)\n",
"print(m)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 208
},
"colab_type": "code",
"id": "AW_RqtcRQFLS",
"outputId": "74a5b955-03e8-494f-e854-f89e49faea86"
},
"outputs": [],
"source": [
"x = torch.rand(5, 4)\n",
"print(x)\n",
"y = m(x)\n",
"print(y)\n",
"print(y.sum(dim=0), y.sum(dim=1))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "bIDwWQ_BQFLT"
},
"source": [
"### Loss function\n",
"- function which tells the model how close its predictions are to the actual values\n",
"- `torch.nn` library has differentloss functions\n",
"\n",
"| loss function | used |\n",
"| --- | --- |\n",
"| L1 loss | Mostly used as a regularizer. |\n",
"| MSE loss | Used as loss function for regression problems. |\n",
"| Cross-entropy loss | Used for binary and multi-class classification problems. |\n",
"| NLL Loss | Used for classification problems and allows us to use specific weights to handle imbalanced datasets. |\n",
"| NLL Loss2d | Used for pixel-wise classification, mostly for problems related to image segmentation. |"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 104
},
"colab_type": "code",
"id": "heJNo6mDQFLT",
"outputId": "6c3487da-012f-4f47-dc1d-0470f0c122e5"
},
"outputs": [],
"source": [
"crit = torch.nn.CrossEntropyLoss()\n",
"input = torch.randn(3, 5, requires_grad=True)\n",
"print(input)\n",
"target = torch.empty(3, dtype=torch.long).random_(5)\n",
"print(target)\n",
"loss = crit(input, target)\n",
"print(loss)\n",
"loss.backward()"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "5OYXl_fiQFLV"
},
"source": [
"### Optimizer\n",
"- optimize the weights to reducethe loss and thus improving the accuracy of the algorithm\n",
"- take loss functions and all the learnable parametersand move them slightly to improve our performances\n",
"- `torch.optim` library has differentloss optimizers: (Adagrad, Adam, SparseAdam, ASGD, SGD)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 237
},
"colab_type": "code",
"id": "FQnkYIFLQFLV",
"outputId": "7a764a77-f13b-49ff-905a-690b772dadde"
},
"outputs": [],
"source": [
"optimizer = optim.Adam(model.parameters(),lr=0.01)\n",
"\n",
"for input, target in dataset:\n",
" optimizer.zero_grad()\n",
" output = model(input)\n",
" loss = loss_fn(output, target)\n",
" loss.backward()\n",
" optimizer.step()"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "uuZEpF3aQFLW"
},
"source": [
"### Overfitting and underfitting\n",
"- Overfitting: algorithm overfits when it performs well on the training dataset but fails to perform on unseen or validation and test datasets - memorize the dataset\n",
" - getting more data\n",
" - reducing the size of the network\n",
" - applying weight regularizer\n",
" - applying dropout\n",
"- Underfitting: model may fail to learn any patterns from our training data\n",
" - getting more data\n",
" - increase the complexity of the model\n",
" - applying weight regularizer"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "KIjJeWRQQFLW"
},
"source": [
"### Dropout\n",
"\n",
"![add image](https://i.imgur.com/AKdSzrX.png)\n",
"\n",
"- one of the most commonly used and the most powerful regularization techniques\n",
"- is applied to intermediate layers of the model during the training time\n",
"- [torch.nn.Dropout](https://pytorch.org/docs/stable/nn.html#torch.nn.Dropout)\n",
"- [torch.nn.functional.dropout()](https://pytorch.org/docs/stable/nn.functional.html#torch.nn.functional.dropout)\n",
"- `torch.nn.Dropout(p=0.2)`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "9-2J8rz1QFLW",
"outputId": "d9e356e0-53d5-43fc-d11f-3cd136bd05cf"
},
"outputs": [],
"source": [
"m = torch.nn.Dropout(p=0.5)\n",
"print(m)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 121
},
"colab_type": "code",
"id": "55bMmsBkQFLY",
"outputId": "7d4ed49f-594f-4a38-f7bd-9f4ee2d6d2e1"
},
"outputs": [],
"source": [
"x = torch.rand(3, 5)\n",
"print(x)\n",
"y = m(x)\n",
"print(y)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "gAKqbWfqQFLZ"
},
"source": [
"### Layer Normalization\n",
"- faster, more stable training\n",
"- [torch.nn.LayerNorm](https://pytorch.org/docs/stable/nn.html#torch.nn.LayerNorm)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "g0b93InjQFLZ",
"outputId": "00025c32-3c06-49a8-88bb-5e0ac9fd3fb3"
},
"outputs": [],
"source": [
"m = torch.nn.LayerNorm(125)\n",
"print(m)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "hgi7xtUWQFLa",
"outputId": "944422f4-17dc-488f-9877-33b329d4ae7f"
},
"outputs": [],
"source": [
"x = torch.rand(15, 125)\n",
"print(x.size())\n",
"y = m(x)\n",
"print(y.size())"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "Ze4fyB-IQFLc"
},
"source": [
"## Module\n",
"- base class for all neural network modules\n",
"- [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "Xw1bv1IfQFLd"
},
"outputs": [],
"source": [
"class MyNet(torch.nn.Module):\n",
" \n",
" def __init__(self, hidden_size: int):\n",
" super().__init__()\n",
" self.hidden_size = hidden_size\n",
" self.l1 = torch.nn.Linear(128, hidden_size)\n",
" self.l2 = torch.nn.Linear(hidden_size, 128)\n",
" \n",
" def forward(self, x):\n",
" y = self.l1(x)\n",
" y = self.l2(y)\n",
" return y\n",
" # or shorter:\n",
" # return self.l2(self.l1(x))"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 86
},
"colab_type": "code",
"id": "bjei6hZLQFLe",
"outputId": "48fb2a3b-78e1-4dd1-cec5-c1ae878c0f47"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MyNet(\n",
" (l1): Linear(in_features=128, out_features=256, bias=True)\n",
" (l2): Linear(in_features=256, out_features=128, bias=True)\n",
")\n"
]
}
],
"source": [
"m = MyNet(256)\n",
"print(m)"
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "315QWfsfQFLf",
"outputId": "98c314a3-dde2-4f49-bea1-fc56c8dce710"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([15, 128]) -> torch.Size([15, 128])\n",
"Equal: False\n"
]
}
],
"source": [
"x = torch.rand(15, 128)\n",
"y = m(x)\n",
"print(x.size(), '->', y.size())\n",
"print('Equal:', torch.equal(x, y))"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 69
},
"colab_type": "code",
"id": "CQpAIoEwQFLg",
"outputId": "062c6272-f76c-4918-da89-acc406bc28e6"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"cpu\n"
]
},
{
"ename": "AssertionError",
"evalue": "\nThe NVIDIA driver on your system is too old (found version 9010).\nPlease update your GPU driver by downloading and installing a new\nversion from the URL: http://www.nvidia.com/Download/index.aspx\nAlternatively, go to: https://pytorch.org to install\na PyTorch version that has been compiled with your version\nof the CUDA driver.",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mAssertionError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-51-2a01d61c9283>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mm\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0ml1\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mweight\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdevice\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mm\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'cuda'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 3\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mm\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0ml1\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mweight\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdevice\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0mm\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'cpu'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mm\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0ml1\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mweight\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdevice\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36mto\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 423\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdevice\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_floating_point\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnon_blocking\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 424\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 425\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_apply\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconvert\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 426\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 427\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mregister_backward_hook\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhook\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_apply\u001b[0;34m(self, fn)\u001b[0m\n\u001b[1;32m 199\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_apply\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 200\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mmodule\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchildren\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 201\u001b[0;31m \u001b[0mmodule\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_apply\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 202\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 203\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mcompute_should_use_set_data\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtensor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtensor_applied\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_apply\u001b[0;34m(self, fn)\u001b[0m\n\u001b[1;32m 221\u001b[0m \u001b[0;31m# `with torch.no_grad():`\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 222\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mno_grad\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 223\u001b[0;31m \u001b[0mparam_applied\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mparam\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 224\u001b[0m \u001b[0mshould_use_set_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcompute_should_use_set_data\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mparam\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparam_applied\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 225\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mshould_use_set_data\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36mconvert\u001b[0;34m(t)\u001b[0m\n\u001b[1;32m 421\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 422\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mconvert\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mt\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 423\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdevice\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_floating_point\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnon_blocking\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 424\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 425\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_apply\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconvert\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/cuda/__init__.py\u001b[0m in \u001b[0;36m_lazy_init\u001b[0;34m()\u001b[0m\n\u001b[1;32m 194\u001b[0m raise RuntimeError(\n\u001b[1;32m 195\u001b[0m \"Cannot re-initialize CUDA in forked subprocess. \" + msg)\n\u001b[0;32m--> 196\u001b[0;31m \u001b[0m_check_driver\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 197\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_C\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_cuda_init\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 198\u001b[0m \u001b[0m_cudart\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_load_cudart\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/cuda/__init__.py\u001b[0m in \u001b[0;36m_check_driver\u001b[0;34m()\u001b[0m\n\u001b[1;32m 108\u001b[0m \u001b[0mAlternatively\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgo\u001b[0m \u001b[0mto\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mhttps\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m//\u001b[0m\u001b[0mpytorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0morg\u001b[0m \u001b[0mto\u001b[0m \u001b[0minstall\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 109\u001b[0m \u001b[0ma\u001b[0m \u001b[0mPyTorch\u001b[0m \u001b[0mversion\u001b[0m \u001b[0mthat\u001b[0m \u001b[0mhas\u001b[0m \u001b[0mbeen\u001b[0m \u001b[0mcompiled\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0myour\u001b[0m \u001b[0mversion\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 110\u001b[0;31m of the CUDA driver.\"\"\".format(str(torch._C._cuda_getDriverVersion())))\n\u001b[0m\u001b[1;32m 111\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 112\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mAssertionError\u001b[0m: \nThe NVIDIA driver on your system is too old (found version 9010).\nPlease update your GPU driver by downloading and installing a new\nversion from the URL: http://www.nvidia.com/Download/index.aspx\nAlternatively, go to: https://pytorch.org to install\na PyTorch version that has been compiled with your version\nof the CUDA driver."
]
}
],
"source": [
"print(m.l1.weight.device)\n",
"m.to('cuda')\n",
"print(m.l1.weight.device)\n",
"m.to('cpu')\n",
"print(m.l1.weight.device)"
]
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 69
},
"colab_type": "code",
"id": "t-EPvKNWQFLi",
"outputId": "63e7d988-1031-4491-b985-a0428cc47cc8"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"True\n",
"False\n",
"True\n"
]
}
],
"source": [
"print(m.training)\n",
"m.eval()\n",
"print(m.training)\n",
"m.train()\n",
"print(m.training)"
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"colab_type": "code",
"id": "6-CpTi9LQFLj",
"outputId": "9d883010-2eea-4288-b5c4-7e0b9178009e"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"odict_keys(['l1.weight', 'l1.bias', 'l2.weight', 'l2.bias'])\n",
"OrderedDict([('l1.weight', tensor([[-0.0193, 0.0075, -0.0066, ..., -0.0232, 0.0126, -0.0706],\n",
" [-0.0387, -0.0597, 0.0671, ..., -0.0475, 0.0777, -0.0336],\n",
" [ 0.0489, 0.0128, -0.0551, ..., 0.0771, -0.0471, 0.0243],\n",
" ...,\n",
" [ 0.0730, 0.0362, -0.0538, ..., 0.0760, -0.0179, 0.0786],\n",
" [-0.0768, -0.0611, -0.0077, ..., 0.0236, -0.0485, 0.0571],\n",
" [-0.0069, 0.0726, 0.0871, ..., 0.0091, 0.0041, -0.0063]])), ('l1.bias', tensor([ 0.0219, -0.0626, -0.0179, 0.0733, -0.0071, -0.0866, 0.0232, -0.0150,\n",
" -0.0016, 0.0404, -0.0839, -0.0130, 0.0267, -0.0126, 0.0706, -0.0795,\n",
" 0.0578, 0.0022, 0.0038, -0.0775, 0.0197, 0.0177, 0.0190, 0.0238,\n",
" -0.0771, -0.0455, -0.0586, 0.0138, -0.0659, -0.0443, -0.0451, 0.0710,\n",
" -0.0820, -0.0724, 0.0570, 0.0868, -0.0540, -0.0570, 0.0128, 0.0389,\n",
" -0.0867, 0.0409, -0.0622, 0.0600, 0.0283, -0.0248, 0.0042, 0.0491,\n",
" 0.0648, 0.0127, 0.0698, -0.0265, 0.0230, -0.0714, 0.0833, -0.0269,\n",
" -0.0736, -0.0079, 0.0478, -0.0519, -0.0653, 0.0830, -0.0777, -0.0535,\n",
" 0.0074, 0.0476, 0.0035, 0.0461, -0.0807, -0.0004, -0.0046, -0.0367,\n",
" 0.0829, 0.0214, -0.0596, -0.0537, -0.0799, 0.0054, 0.0637, -0.0382,\n",
" 0.0244, -0.0719, 0.0047, 0.0395, 0.0764, 0.0672, -0.0340, -0.0775,\n",
" -0.0073, 0.0047, -0.0576, 0.0221, -0.0596, 0.0799, 0.0635, 0.0517,\n",
" 0.0770, -0.0358, 0.0741, -0.0596, -0.0574, -0.0674, -0.0645, -0.0172,\n",
" 0.0205, -0.0760, -0.0349, -0.0817, 0.0246, 0.0328, 0.0428, -0.0154,\n",
" 0.0564, 0.0180, -0.0686, 0.0088, 0.0675, 0.0701, -0.0609, 0.0883,\n",
" 0.0677, -0.0176, -0.0310, -0.0428, -0.0817, -0.0302, -0.0433, -0.0570,\n",
" -0.0419, -0.0522, 0.0170, 0.0421, 0.0672, -0.0036, 0.0270, 0.0491,\n",
" 0.0763, -0.0390, -0.0704, 0.0649, 0.0582, 0.0628, -0.0334, 0.0134,\n",
" 0.0809, 0.0057, -0.0384, -0.0295, -0.0106, 0.0512, -0.0091, -0.0276,\n",
" 0.0173, 0.0516, -0.0086, 0.0867, -0.0710, 0.0380, -0.0555, 0.0762,\n",
" -0.0620, -0.0151, 0.0367, 0.0822, 0.0403, -0.0171, 0.0730, 0.0584,\n",
" -0.0025, 0.0582, 0.0745, -0.0616, -0.0494, 0.0356, -0.0272, -0.0097,\n",
" 0.0362, 0.0616, -0.0310, 0.0454, -0.0683, 0.0661, -0.0323, 0.0215,\n",
" 0.0375, -0.0673, 0.0543, -0.0783, 0.0621, 0.0392, -0.0194, 0.0852,\n",
" -0.0134, 0.0145, 0.0352, -0.0670, 0.0305, 0.0383, 0.0451, -0.0536,\n",
" 0.0292, 0.0795, 0.0644, 0.0250, 0.0491, 0.0733, -0.0811, -0.0167,\n",
" -0.0517, 0.0010, -0.0524, -0.0370, 0.0741, -0.0795, -0.0542, -0.0835,\n",
" -0.0283, -0.0670, -0.0675, 0.0730, -0.0478, 0.0598, -0.0862, 0.0030,\n",
" 0.0799, -0.0124, -0.0847, -0.0193, -0.0309, -0.0768, 0.0870, 0.0615,\n",
" 0.0764, 0.0399, -0.0713, 0.0800, -0.0873, -0.0523, 0.0093, 0.0752,\n",
" 0.0873, 0.0189, -0.0734, 0.0567, -0.0150, 0.0674, 0.0450, 0.0549,\n",
" 0.0032, -0.0810, -0.0867, 0.0349, -0.0298, -0.0467, 0.0489, 0.0385])), ('l2.weight', tensor([[-1.4906e-02, -8.1936e-03, -3.2866e-02, ..., 4.1430e-02,\n",
" -2.1258e-02, -6.2108e-03],\n",
" [ 8.5995e-05, -2.4441e-02, -9.7788e-03, ..., 8.8740e-03,\n",
" -1.8025e-02, -3.1464e-02],\n",
" [-5.0128e-02, -3.9290e-02, 2.6125e-02, ..., -3.5566e-02,\n",
" -4.6239e-02, 4.8540e-02],\n",
" ...,\n",
" [ 1.8081e-02, 3.0562e-02, 5.0640e-03, ..., 9.6664e-03,\n",
" 1.3690e-02, -3.0873e-02],\n",
" [-1.5538e-02, -5.4558e-02, 5.5808e-02, ..., 7.9329e-03,\n",
" 5.5620e-02, -8.7916e-03],\n",
" [-2.6634e-02, -3.8213e-02, -8.9528e-04, ..., -5.4874e-02,\n",
" 6.0545e-02, -2.8823e-02]])), ('l2.bias', tensor([-0.0385, -0.0052, 0.0611, 0.0359, 0.0577, 0.0022, 0.0614, -0.0162,\n",
" 0.0405, 0.0241, -0.0551, -0.0186, 0.0609, 0.0239, 0.0333, 0.0167,\n",
" 0.0574, -0.0497, 0.0081, -0.0538, -0.0290, 0.0122, 0.0559, 0.0121,\n",
" 0.0037, 0.0438, -0.0325, -0.0232, 0.0503, 0.0216, 0.0560, -0.0532,\n",
" -0.0544, -0.0444, 0.0395, -0.0268, -0.0306, -0.0416, 0.0374, -0.0321,\n",
" -0.0097, 0.0421, 0.0120, 0.0322, 0.0446, -0.0325, -0.0331, 0.0467,\n",
" 0.0436, -0.0291, 0.0013, -0.0409, -0.0135, 0.0585, 0.0115, -0.0048,\n",
" -0.0196, 0.0101, 0.0267, -0.0068, 0.0485, 0.0515, 0.0018, -0.0068,\n",
" 0.0313, 0.0185, 0.0315, 0.0189, -0.0334, 0.0343, -0.0188, -0.0137,\n",
" -0.0445, 0.0180, -0.0425, 0.0154, 0.0511, 0.0175, 0.0463, 0.0346,\n",
" 0.0139, 0.0059, -0.0299, -0.0025, 0.0011, -0.0146, -0.0022, -0.0430,\n",
" 0.0537, -0.0359, -0.0197, 0.0575, -0.0334, 0.0018, -0.0588, -0.0273,\n",
" 0.0210, -0.0210, -0.0481, -0.0075, 0.0230, -0.0032, 0.0305, -0.0393,\n",
" -0.0206, 0.0116, 0.0087, 0.0124, 0.0216, 0.0537, 0.0299, -0.0616,\n",
" -0.0033, 0.0413, 0.0301, -0.0422, -0.0118, -0.0486, 0.0547, 0.0172,\n",
" 0.0229, -0.0008, -0.0322, -0.0608, -0.0275, -0.0594, -0.0516, 0.0502]))])\n"
]
}
],
"source": [
"print(m.state_dict().keys())\n",
"print(m.state_dict())"
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"colab_type": "code",
"id": "iiO5FCaOQFLk",
"outputId": "91446e61-b785-4177-d336-38c06fff2818"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'config': {'hidden_size': 256}, 'state_dict': OrderedDict([('l1.weight', tensor([[-0.0193, 0.0075, -0.0066, ..., -0.0232, 0.0126, -0.0706],\n",
" [-0.0387, -0.0597, 0.0671, ..., -0.0475, 0.0777, -0.0336],\n",
" [ 0.0489, 0.0128, -0.0551, ..., 0.0771, -0.0471, 0.0243],\n",
" ...,\n",
" [ 0.0730, 0.0362, -0.0538, ..., 0.0760, -0.0179, 0.0786],\n",
" [-0.0768, -0.0611, -0.0077, ..., 0.0236, -0.0485, 0.0571],\n",
" [-0.0069, 0.0726, 0.0871, ..., 0.0091, 0.0041, -0.0063]])), ('l1.bias', tensor([ 0.0219, -0.0626, -0.0179, 0.0733, -0.0071, -0.0866, 0.0232, -0.0150,\n",
" -0.0016, 0.0404, -0.0839, -0.0130, 0.0267, -0.0126, 0.0706, -0.0795,\n",
" 0.0578, 0.0022, 0.0038, -0.0775, 0.0197, 0.0177, 0.0190, 0.0238,\n",
" -0.0771, -0.0455, -0.0586, 0.0138, -0.0659, -0.0443, -0.0451, 0.0710,\n",
" -0.0820, -0.0724, 0.0570, 0.0868, -0.0540, -0.0570, 0.0128, 0.0389,\n",
" -0.0867, 0.0409, -0.0622, 0.0600, 0.0283, -0.0248, 0.0042, 0.0491,\n",
" 0.0648, 0.0127, 0.0698, -0.0265, 0.0230, -0.0714, 0.0833, -0.0269,\n",
" -0.0736, -0.0079, 0.0478, -0.0519, -0.0653, 0.0830, -0.0777, -0.0535,\n",
" 0.0074, 0.0476, 0.0035, 0.0461, -0.0807, -0.0004, -0.0046, -0.0367,\n",
" 0.0829, 0.0214, -0.0596, -0.0537, -0.0799, 0.0054, 0.0637, -0.0382,\n",
" 0.0244, -0.0719, 0.0047, 0.0395, 0.0764, 0.0672, -0.0340, -0.0775,\n",
" -0.0073, 0.0047, -0.0576, 0.0221, -0.0596, 0.0799, 0.0635, 0.0517,\n",
" 0.0770, -0.0358, 0.0741, -0.0596, -0.0574, -0.0674, -0.0645, -0.0172,\n",
" 0.0205, -0.0760, -0.0349, -0.0817, 0.0246, 0.0328, 0.0428, -0.0154,\n",
" 0.0564, 0.0180, -0.0686, 0.0088, 0.0675, 0.0701, -0.0609, 0.0883,\n",
" 0.0677, -0.0176, -0.0310, -0.0428, -0.0817, -0.0302, -0.0433, -0.0570,\n",
" -0.0419, -0.0522, 0.0170, 0.0421, 0.0672, -0.0036, 0.0270, 0.0491,\n",
" 0.0763, -0.0390, -0.0704, 0.0649, 0.0582, 0.0628, -0.0334, 0.0134,\n",
" 0.0809, 0.0057, -0.0384, -0.0295, -0.0106, 0.0512, -0.0091, -0.0276,\n",
" 0.0173, 0.0516, -0.0086, 0.0867, -0.0710, 0.0380, -0.0555, 0.0762,\n",
" -0.0620, -0.0151, 0.0367, 0.0822, 0.0403, -0.0171, 0.0730, 0.0584,\n",
" -0.0025, 0.0582, 0.0745, -0.0616, -0.0494, 0.0356, -0.0272, -0.0097,\n",
" 0.0362, 0.0616, -0.0310, 0.0454, -0.0683, 0.0661, -0.0323, 0.0215,\n",
" 0.0375, -0.0673, 0.0543, -0.0783, 0.0621, 0.0392, -0.0194, 0.0852,\n",
" -0.0134, 0.0145, 0.0352, -0.0670, 0.0305, 0.0383, 0.0451, -0.0536,\n",
" 0.0292, 0.0795, 0.0644, 0.0250, 0.0491, 0.0733, -0.0811, -0.0167,\n",
" -0.0517, 0.0010, -0.0524, -0.0370, 0.0741, -0.0795, -0.0542, -0.0835,\n",
" -0.0283, -0.0670, -0.0675, 0.0730, -0.0478, 0.0598, -0.0862, 0.0030,\n",
" 0.0799, -0.0124, -0.0847, -0.0193, -0.0309, -0.0768, 0.0870, 0.0615,\n",
" 0.0764, 0.0399, -0.0713, 0.0800, -0.0873, -0.0523, 0.0093, 0.0752,\n",
" 0.0873, 0.0189, -0.0734, 0.0567, -0.0150, 0.0674, 0.0450, 0.0549,\n",
" 0.0032, -0.0810, -0.0867, 0.0349, -0.0298, -0.0467, 0.0489, 0.0385])), ('l2.weight', tensor([[-1.4906e-02, -8.1936e-03, -3.2866e-02, ..., 4.1430e-02,\n",
" -2.1258e-02, -6.2108e-03],\n",
" [ 8.5995e-05, -2.4441e-02, -9.7788e-03, ..., 8.8740e-03,\n",
" -1.8025e-02, -3.1464e-02],\n",
" [-5.0128e-02, -3.9290e-02, 2.6125e-02, ..., -3.5566e-02,\n",
" -4.6239e-02, 4.8540e-02],\n",
" ...,\n",
" [ 1.8081e-02, 3.0562e-02, 5.0640e-03, ..., 9.6664e-03,\n",
" 1.3690e-02, -3.0873e-02],\n",
" [-1.5538e-02, -5.4558e-02, 5.5808e-02, ..., 7.9329e-03,\n",
" 5.5620e-02, -8.7916e-03],\n",
" [-2.6634e-02, -3.8213e-02, -8.9528e-04, ..., -5.4874e-02,\n",
" 6.0545e-02, -2.8823e-02]])), ('l2.bias', tensor([-0.0385, -0.0052, 0.0611, 0.0359, 0.0577, 0.0022, 0.0614, -0.0162,\n",
" 0.0405, 0.0241, -0.0551, -0.0186, 0.0609, 0.0239, 0.0333, 0.0167,\n",
" 0.0574, -0.0497, 0.0081, -0.0538, -0.0290, 0.0122, 0.0559, 0.0121,\n",
" 0.0037, 0.0438, -0.0325, -0.0232, 0.0503, 0.0216, 0.0560, -0.0532,\n",
" -0.0544, -0.0444, 0.0395, -0.0268, -0.0306, -0.0416, 0.0374, -0.0321,\n",
" -0.0097, 0.0421, 0.0120, 0.0322, 0.0446, -0.0325, -0.0331, 0.0467,\n",
" 0.0436, -0.0291, 0.0013, -0.0409, -0.0135, 0.0585, 0.0115, -0.0048,\n",
" -0.0196, 0.0101, 0.0267, -0.0068, 0.0485, 0.0515, 0.0018, -0.0068,\n",
" 0.0313, 0.0185, 0.0315, 0.0189, -0.0334, 0.0343, -0.0188, -0.0137,\n",
" -0.0445, 0.0180, -0.0425, 0.0154, 0.0511, 0.0175, 0.0463, 0.0346,\n",
" 0.0139, 0.0059, -0.0299, -0.0025, 0.0011, -0.0146, -0.0022, -0.0430,\n",
" 0.0537, -0.0359, -0.0197, 0.0575, -0.0334, 0.0018, -0.0588, -0.0273,\n",
" 0.0210, -0.0210, -0.0481, -0.0075, 0.0230, -0.0032, 0.0305, -0.0393,\n",
" -0.0206, 0.0116, 0.0087, 0.0124, 0.0216, 0.0537, 0.0299, -0.0616,\n",
" -0.0033, 0.0413, 0.0301, -0.0422, -0.0118, -0.0486, 0.0547, 0.0172,\n",
" 0.0229, -0.0008, -0.0322, -0.0608, -0.0275, -0.0594, -0.0516, 0.0502]))])}\n"
]
}
],
"source": [
"model_data = {\n",
" 'config': {\n",
" 'hidden_size': m.hidden_size,\n",
" },\n",
" 'state_dict': m.state_dict(),\n",
"}\n",
"print(model_data)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "WQKiLfwIQFLm"
},
"source": [
"[torch.save()](https://pytorch.org/docs/stable/torch.html#torch.save) [torch.load()](https://pytorch.org/docs/stable/torch.html#torch.load)"
]
},
{
"cell_type": "code",
"execution_count": 55,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "xG9REjM-QFLm"
},
"outputs": [],
"source": [
"torch.save(model_data, 'my_model.bin')"
]
},
{
"cell_type": "code",
"execution_count": 56,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "nSYqOM8RQFLn",
"outputId": "0731fabf-a96f-4ed0-9306-a120143382e7"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"-rw-r--r-- 1 pawel pawel 259K maj 9 16:29 my_model.bin\r\n"
]
}
],
"source": [
"!ls -lh my_model.bin"
]
},
{
"cell_type": "code",
"execution_count": 57,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "VaBqd0-VQFLp",
"outputId": "e33a6b8e-9a81-4510-e2c5-87c3259802de"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"dict_keys(['config', 'state_dict'])\n",
"{'hidden_size': 256}\n"
]
}
],
"source": [
"loaded_data = torch.load('my_model.bin', map_location=torch.device('cpu'))\n",
"print(loaded_data.keys())\n",
"print(loaded_data['config'])"
]
},
{
"cell_type": "code",
"execution_count": 58,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 104
},
"colab_type": "code",
"id": "juJJzC4PQFLp",
"outputId": "c64172cd-9b5f-4fec-aaaa-9bfd41995b09"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MyNet(\n",
" (l1): Linear(in_features=128, out_features=256, bias=True)\n",
" (l2): Linear(in_features=256, out_features=128, bias=True)\n",
")\n"
]
},
{
"data": {
"text/plain": [
"<All keys matched successfully>"
]
},
"execution_count": 58,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m = MyNet(**loaded_data['config'])\n",
"print(m)\n",
"m.load_state_dict(loaded_data['state_dict'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "ptZbbMCaQFLr"
},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "w7hCRHBPQFLr"
},
"outputs": [],
"source": [
"def init_weight(my_net: MyNet, uniform: bool = False):\n",
" if uniform:\n",
" torch.nn.init.xavier_uniform_(my_net.l1.weight)\n",
" torch.nn.init.xavier_uniform_(my_net.l2.weight)\n",
" else:\n",
" torch.nn.init.xavier_normal_(my_net.l1.weight)\n",
" torch.nn.init.xavier_normal_(my_net.l2.weight)\n",
" torch.nn.init.zeros_(my_net.l1.bias)\n",
" torch.nn.init.zeros_(my_net.l2.bias)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 104
},
"colab_type": "code",
"id": "vvIhYQL5QFLt",
"outputId": "dcf20b94-ff1e-48f4-d670-3e368b8d1440"
},
"outputs": [],
"source": [
"m1 = MyNet(256).eval()\n",
"init_weight(m1)\n",
"print(m1)\n",
"\n",
"x1 = torch.rand(15, 128)\n",
"\n",
"y1 = m1(x1)\n",
"y2 = m1(x1)\n",
"print('y1 == y2:', torch.equal(y1, y2))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "wfTVWm6iQFLw"
},
"outputs": [],
"source": [
"model_data = {\n",
" 'config': {\n",
" 'hidden_size': m1.hidden_size,\n",
" },\n",
" 'state_dict': m1.state_dict(),\n",
"}\n",
"torch.save(model_data, 'my_model.bin')\n",
"loaded_data = torch.load('my_model.bin', map_location=torch.device('cpu'))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 121
},
"colab_type": "code",
"id": "ugcAIFfOQFLw",
"outputId": "f6518e6f-99e6-4a77-b8ad-506cff08f1ed"
},
"outputs": [],
"source": [
"m2 = MyNet(256).eval()\n",
"init_weight(m2, uniform=True)\n",
"print(m2)\n",
"\n",
"z1 = m2(x1)\n",
"z2 = m2(x1)\n",
"print('z1 == z2:', torch.equal(z1, z2))\n",
"print('z1 == y1:', torch.equal(z1, y1))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 121
},
"colab_type": "code",
"id": "5C-3xPEhQFLy",
"outputId": "e262adfd-c00d-4024-cb2f-bf2eac60e7aa"
},
"outputs": [],
"source": [
"m2.load_state_dict(loaded_data['state_dict'])\n",
"print(m2)\n",
"\n",
"z1 = m2(x1)\n",
"z2 = m2(x1)\n",
"print('z1 == z2:', torch.equal(z1, z2))\n",
"print('z1 == y1:', torch.equal(z1, y1))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "2BWpYk_1QFL9"
},
"outputs": [],
"source": [
"def almost_equal_tensors(x: torch.Tensor, y: torch.Tensor, epsilon: float = 1e-12) -> bool:\n",
" if list(x.size()) != list(y.size()):\n",
" print(f'Invalid tensor size: x: {list(x.size())} and y: {list(y.size())}')\n",
" return False\n",
"\n",
" result_comp = torch.lt(torch.abs(x - y), epsilon)\n",
" is_equal = bool(torch.all(result_comp))\n",
" if not is_equal:\n",
" # Count number of elements\n",
" total_all = result_comp.numel()\n",
" total_true = torch.sum(result_comp)\n",
" percentage = float(total_true) / float(total_all) * 100.0\n",
" print(f'Tensor are not equal with epsilon {epsilon} ({percentage:.2f}% correct)')\n",
" return False\n",
" return True"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 121
},
"colab_type": "code",
"id": "QVlK3YTaQFL-",
"outputId": "59e77190-cf4c-4db2-a0cc-acff6520f2f5"
},
"outputs": [],
"source": [
"print(z1[0][:5])\n",
"print(y1[0][:5])\n",
"\n",
"print(almost_equal_tensors(z1, y1))\n",
"\n",
"print(almost_equal_tensors(z1, y1 - 1e-12))\n",
"\n",
"print(almost_equal_tensors(z1, y1 - 1e-8))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "5yuvtHpGQFMA"
},
"source": [
"---\n",
"### Zadania:"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "vFje-GcFQFMA"
},
"source": [
"**Zadanie 1**:\n",
"\n",
"Napisz funkcję, która dla wektora obliczy odpowiednio sinus i cosinus według wzoru:\n",
"- dla indeksów parzystych obliczy sinus\n",
"- dla indeksów nieparzysztych obliczy cosinus"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "YbDIxhwHQFMA"
},
"outputs": [],
"source": [
"def function_1(x: torch.FloatTensor) -> torch.FloatTensor:\n",
" x[::2] = x[::2].cos()\n",
" x[1::2] = x[1::2].sin()\n",
" return x"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "Hl6jWe2cQFMB",
"outputId": "3d0f4da9-e0be-4e4c-8bb4-2d45c8a5f08f"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Testing example 0\n",
"Testing example 1\n"
]
}
],
"source": [
"for i, x in enumerate([torch.FloatTensor([1, 2, 3]), torch.FloatTensor([23, 25, 33, 14, 85])]):\n",
" print(f'Testing example {i}')\n",
" y = function_1(x)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "sZ50m4ptQFMC"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "2vc9RHk9QFMD"
},
"source": [
"**Zadanie 2**:\n",
"\n",
"Napisz funckję, która dla dwóch wektorów obliczy wyrażenie: `2.5*x + y^3 - mean(x, y)`\n",
"\n",
"Uwaga: dla operacji `mean(x, y)` można wykorzystać polecenia: `torch.mean()` oraz `torch.stack`"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "XoJYjvYhQFMD"
},
"outputs": [],
"source": [
"def function_2(x: torch.FloatTensor, y: torch.FloatTensor) -> torch.FloatTensor:\n",
" return 2.5*x+y**3-torch.mean(torch.stack([x,y]))"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"colab_type": "code",
"id": "OAuFkF1SQFMF",
"outputId": "93ec5f29-6278-4145-cfff-05470e64fb0b"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Testing example 0\n",
"Testing example 1\n"
]
}
],
"source": [
"for i, (x, y) in enumerate([(torch.FloatTensor([1, 2, 3]), torch.FloatTensor([3, 2, 1])),\n",
" (torch.FloatTensor([1, 1, 1, 1, 1]), torch.FloatTensor([8, 4, 123, 16, 12]))]):\n",
" print(f'Testing example {i}')\n",
" z = function_2(x, y)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "ye8QVb3RQFMI"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "TLkroubGQFMJ"
},
"source": [
"**Zadanie 3**:\n",
"\n",
"Wybierz najbardziej podobne 2 wektory do podanego na wejściu z dostępnego Tensora (dostępnych wektorów).\n",
"\n",
"Uwaga: Do obliczenia podobieństwa wektorów można wykorzystać **odległość euklidesową** lub inną (np. odległość kosinusową - `torch.nn.functional.cosine_similarity` - dla wektorów 1D należy wykorzystać argument `dim=0`)."
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "hV1t6z1uQFMJ"
},
"outputs": [],
"source": [
"def function_3(vectors: torch.FloatTensor, search: torch.FloatTensor) -> torch.FloatTensor:\n",
" result = []\n",
" for vector in vectors:\n",
" result.append(torch.nn.functional.cosine_similarity(vector, search, dim=0))\n",
" vec1 = torch.tensor(result).argmax()\n",
" result[vec1] = 0\n",
" vec2 = torch.tensor(result).argmax()\n",
" return [vectors[vec1], vectors[vec2]]\n",
" "
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "mOoloxI6QFMK",
"outputId": "ddaf9ef4-c975-4e82-83c7-a4545820df56",
"scrolled": true
},
"outputs": [],
"source": [
"for i, (x, y) in enumerate([(torch.FloatTensor([[2, 2, 2], [1, 2, 1], [1, 0, 3], [123, 1, 1]]), torch.FloatTensor([1, 1, 1]))]):\n",
" z = function_3(x, y)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "PvMnyUKHQFML"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "-J2jabEvQFMM"
},
"source": [
"**Zadanie 4**:\n",
"\n",
"Zapisz dowolny element do pliku z wykorzystaniem biblioteki `pickle`, następnie usuń zapisany element z pamięci wykorzystując fukcję `del`, kolejno wczytaj zapisane dane i zmodyfikuj je w dowolny sposób."
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<class 'dict'> {'hello': 'world', 'a': 1, 'b': 2}\n"
]
}
],
"source": [
"import pickle\n",
"\n",
"vocab = {'hello': 'world', 'a': 1, 'b': 2}\n",
"\n",
"with open('plik.pkl', 'wb') as f:\n",
" pickle.dump(vocab, f)\n",
" \n",
"del vocab\n",
"\n",
"with open('plik.pkl', 'rb') as f:\n",
" vocab = pickle.load(f)\n",
"\n",
"print(type(vocab), vocab)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "f5dZlRwMQFMN"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "f5Xet4xGQFMN"
},
"source": [
"**Zadanie 5**:\n",
"\n",
"Napisz Moduł (`torch.nn.Module`) zawierający elementy:\n",
"- emb: `torch.nn.Embedding` (num_tokens, emb_in)\n",
"- norm: `torch.nn.LayerNorm` (emb_in)\n",
"- act: `torch.nn.ReLU`\n",
"- dropout: `torch.nn.Dropout`\n",
"- out: `torch.nn.Linear` (emb_in, emb_out)\n",
"\n",
"Proces przetwarzania danych powinien przebiegać według kolejności elementów przedstawionych powyżej.\n",
"\n",
"Moduł powinien być konfigurowalny przy pomocy argumenty argumnetów:\n",
"- emb_in: `int` - wielkość embeddingów wejściowych\n",
"- emb_out: `int` - wielkość embeddingów wyjściowych (wymiar danych po wyjściu z sieci)\n",
"- num_tokens: `int` - maksymalna wielkość/ilość tokenów (wielkość słownika) dla warstwy Embeddings\n",
"- dropout: `float` - wielkośc/wartość dropout"
]
},
{
"cell_type": "code",
"execution_count": 65,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "MWrWCSMFQFMN"
},
"outputs": [],
"source": [
"class MyModule(torch.nn.Module):\n",
" \n",
" def __init__(self, emb_in: int, emb_out: int, num_tokens: int, dropout: float):\n",
" super().__init__()\n",
" \n",
" self.emb_in = emb_in\n",
" self.emb_out = emb_out\n",
" self.num_tokens = num_tokens\n",
" self.dropout = dropout\n",
" \n",
" self.emb = torch.nn.Embedding(num_tokens, emb_in)\n",
" self.norm = torch.nn.LayerNorm(emb_in)\n",
" self.act = torch.nn.ReLU()\n",
" self.dropout = torch.nn.Dropout()\n",
" self.out = torch.nn.Linear(emb_in, emb_out)\n",
" \n",
" def forward(self, x: torch.LongTensor) -> torch.FloatTensor:\n",
" x = self.emb(x)\n",
" x = self.norm(x)\n",
" x = self.act(x)\n",
" x = self.dropout(x)\n",
" return self.out(x)"
]
},
{
"cell_type": "code",
"execution_count": 66,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "lPgqpN3KQFMO",
"outputId": "9bda1ccf-ca02-4efb-9ddc-7247dd0cdd00"
},
"outputs": [],
"source": [
"m = MyModule(128, 256, 15000, 0.1)\n",
"x = torch.LongTensor(15, 10).random_(0, 15000)\n",
"y = m(x)\n",
"if [15, 10, 256] != list(y.size()):\n",
" print('Invalid out size!')"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "S5N2D8CwQFMQ"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "M-013eWLQFMQ"
},
"source": [
"**Zadanie 6**:\n",
"\n",
"Zapisz wcześniej utworzony moduł (`state_dict`) do pliku, następnie utworzyć nowy moduł i wczytać zapisany stan (`state_dicy`) z pliku."
]
},
{
"cell_type": "code",
"execution_count": 69,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'config': {'emb_in': 128, 'emb_out': 256, 'num_tokens': 15000, 'dropout': Dropout(p=0.5, inplace=False)}, 'state_dict': OrderedDict([('emb.weight', tensor([[ 0.3181, 0.5409, 0.2802, ..., 1.4907, -0.8508, -0.3714],\n",
" [-1.4543, -1.9252, -0.2154, ..., -1.7649, -0.3797, 1.3157],\n",
" [ 0.4123, 0.0904, 0.3782, ..., 0.2839, -1.3487, -0.4568],\n",
" ...,\n",
" [-0.2046, -1.8456, -1.0061, ..., -1.7294, 1.0968, 1.2567],\n",
" [ 0.3166, 0.1134, 0.4829, ..., 0.5419, 1.8218, 0.1062],\n",
" [ 0.1806, 1.1378, -0.7690, ..., -0.8517, -0.0662, 2.5349]])), ('norm.weight', tensor([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
" 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
" 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
" 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
" 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
" 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
" 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
" 1., 1.])), ('norm.bias', tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
" 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
" 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
" 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
" 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n",
" 0., 0., 0., 0., 0., 0., 0., 0.])), ('out.weight', tensor([[-0.0427, 0.0441, -0.0778, ..., -0.0384, 0.0252, -0.0082],\n",
" [ 0.0101, -0.0006, 0.0543, ..., 0.0639, -0.0532, -0.0378],\n",
" [-0.0627, 0.0415, 0.0211, ..., -0.0243, -0.0347, 0.0444],\n",
" ...,\n",
" [-0.0765, -0.0017, -0.0058, ..., -0.0366, 0.0115, 0.0024],\n",
" [ 0.0844, 0.0104, -0.0770, ..., -0.0650, 0.0419, -0.0571],\n",
" [-0.0199, 0.0623, 0.0472, ..., 0.0586, -0.0665, 0.0717]])), ('out.bias', tensor([ 0.0807, 0.0538, 0.0808, 0.0798, -0.0867, 0.0364, -0.0338, -0.0750,\n",
" -0.0624, 0.0578, -0.0595, -0.0519, 0.0242, 0.0341, -0.0046, 0.0034,\n",
" 0.0744, -0.0828, 0.0522, -0.0591, 0.0234, -0.0155, -0.0669, 0.0694,\n",
" 0.0234, -0.0069, -0.0470, 0.0323, 0.0801, 0.0048, -0.0771, 0.0097,\n",
" 0.0318, 0.0041, -0.0518, -0.0077, -0.0581, 0.0649, -0.0353, -0.0339,\n",
" 0.0074, 0.0524, -0.0384, 0.0252, 0.0834, -0.0164, -0.0127, -0.0818,\n",
" -0.0795, -0.0346, -0.0247, -0.0263, -0.0645, -0.0171, -0.0791, 0.0316,\n",
" 0.0246, 0.0514, 0.0805, -0.0352, -0.0384, 0.0822, 0.0143, -0.0194,\n",
" 0.0301, 0.0234, 0.0837, -0.0359, -0.0755, -0.0720, 0.0205, 0.0510,\n",
" -0.0822, -0.0345, -0.0450, -0.0865, 0.0097, 0.0470, -0.0424, 0.0592,\n",
" 0.0839, 0.0185, 0.0343, -0.0713, -0.0009, -0.0313, -0.0393, -0.0536,\n",
" 0.0303, 0.0108, -0.0079, -0.0525, -0.0172, 0.0028, -0.0088, -0.0139,\n",
" -0.0037, 0.0557, 0.0604, 0.0566, 0.0159, 0.0787, -0.0550, -0.0650,\n",
" -0.0139, 0.0630, 0.0632, -0.0340, 0.0779, -0.0861, 0.0806, -0.0525,\n",
" -0.0542, 0.0599, 0.0547, -0.0270, 0.0259, 0.0817, -0.0758, -0.0635,\n",
" 0.0197, 0.0381, 0.0670, -0.0456, 0.0132, -0.0556, 0.0806, 0.0556,\n",
" -0.0370, 0.0374, -0.0303, 0.0883, -0.0455, -0.0288, -0.0531, -0.0717,\n",
" 0.0360, 0.0851, 0.0176, -0.0102, -0.0685, 0.0219, -0.0863, -0.0369,\n",
" 0.0194, 0.0326, -0.0380, -0.0864, 0.0331, 0.0660, -0.0605, -0.0727,\n",
" -0.0422, -0.0582, 0.0377, -0.0252, 0.0390, -0.0279, 0.0408, -0.0144,\n",
" 0.0370, -0.0067, 0.0663, 0.0460, -0.0657, -0.0307, -0.0663, -0.0678,\n",
" -0.0232, -0.0658, 0.0275, 0.0439, 0.0601, 0.0507, -0.0051, 0.0671,\n",
" -0.0292, 0.0251, -0.0597, -0.0330, -0.0197, 0.0592, 0.0262, -0.0020,\n",
" 0.0717, -0.0775, -0.0040, 0.0712, 0.0472, -0.0311, 0.0785, 0.0867,\n",
" -0.0761, 0.0844, 0.0460, -0.0339, 0.0648, 0.0578, 0.0248, -0.0784,\n",
" -0.0087, 0.0319, -0.0527, -0.0848, -0.0872, 0.0444, -0.0635, 0.0744,\n",
" 0.0140, -0.0307, 0.0418, -0.0270, -0.0013, -0.0796, -0.0141, 0.0744,\n",
" 0.0279, 0.0577, 0.0059, -0.0824, 0.0754, 0.0733, 0.0683, 0.0858,\n",
" 0.0138, -0.0350, 0.0658, 0.0437, -0.0827, -0.0066, 0.0419, -0.0661,\n",
" 0.0656, -0.0691, 0.0229, 0.0016, 0.0524, 0.0430, 0.0787, -0.0427,\n",
" -0.0234, 0.0625, 0.0276, -0.0373, -0.0679, 0.0328, -0.0031, 0.0240,\n",
" 0.0105, 0.0065, 0.0169, -0.0341, -0.0776, 0.0261, 0.0629, -0.0517]))])}\n"
]
}
],
"source": [
"model_data2 = {\n",
" 'config': {\n",
" 'emb_in': m.emb_in,\n",
" 'emb_out': m.emb_out,\n",
" 'num_tokens': m.num_tokens,\n",
" 'dropout': m.dropout\n",
" },\n",
" 'state_dict': m.state_dict(),\n",
"}\n",
"print(model_data2)"
]
},
{
"cell_type": "code",
"execution_count": 70,
"metadata": {},
"outputs": [],
"source": [
"torch.save(model_data2, 'my_model2.bin')"
]
},
{
"cell_type": "code",
"execution_count": 73,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MyModule(\n",
" (dropout): Dropout(p=0.5, inplace=False)\n",
" (emb): Embedding(15000, 128)\n",
" (norm): LayerNorm((128,), eps=1e-05, elementwise_affine=True)\n",
" (act): ReLU()\n",
" (out): Linear(in_features=128, out_features=256, bias=True)\n",
")\n"
]
},
{
"data": {
"text/plain": [
"<All keys matched successfully>"
]
},
"execution_count": 73,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"loaded_data2 = torch.load('my_model2.bin', map_location=torch.device('cpu'))\n",
"m2 = MyModule(**loaded_data2['config'])\n",
"print(m2)\n",
"m2.load_state_dict(loaded_data2['state_dict'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "qrMmhG4RQFMQ"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "0btApk6zQFMR"
},
"source": [
"*Uwagi:*\n",
"\n",
"- Aby otrzymać dostęp do **karty graficznej/obliczeń CUDA** w *Google Colab* należy kliknąć z menu u góry **`Runtime`**, po czym wybrać **`Change runtime type`** i w menu **`Hardware accelerator`** zaznaczyć **`GPU`**. Po przetworzeniu, karta graficzna/CUDA powinna być dostępna. **UWAGA:** po zmiane typu urządzanie należy wykonać część kroków ponownie (np. importowanie bibliotek, inicjalizacja zmiennych, czy definicja klas/funkcji), usunięte zostaną również wszytkie wcześniej utworzone dane."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "vJOJyd1LlNeY"
},
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [
"GSGSoTygQFK9",
"c5OMEYrqQFLK",
"-b_yllS6QFLQ",
"uuZEpF3aQFLW"
],
"name": "Copy of 1 PyTorch.ipynb",
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
"02b9468a1d214bc08a7f2c3b0fb0be62": {
"model_module": "@jupyter-widgets/controls",
"model_name": "HBoxModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HBoxModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HBoxView",
"box_style": "",
"children": [
"IPY_MODEL_217de6953d134b028643fe7278574ec0",
"IPY_MODEL_30b59bbc8c9c46bdb03207ea0270e086"
],
"layout": "IPY_MODEL_e3daf6ee9be947c385914a74cb51bac9"
}
},
"0b4166703dc6439bb2534e23e3ab9921": {
"model_module": "@jupyter-widgets/controls",
"model_name": "ProgressStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "ProgressStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"bar_color": null,
"description_width": "initial"
}
},
"16d85c7b33f1437a965dbab57d3d5c79": {
"model_module": "@jupyter-widgets/controls",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"2029f5210be64624bfda9b8dbd4b623c": {
"model_module": "@jupyter-widgets/controls",
"model_name": "HBoxModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HBoxModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HBoxView",
"box_style": "",
"children": [
"IPY_MODEL_dfd11ed9a4bc469ca95a22c0fcad7d05",
"IPY_MODEL_9885433b7d4f407582abc288b9dabfdf"
],
"layout": "IPY_MODEL_85f2503489d24c2297c3589a806c89c1"
}
},
"206a29154ce846568cc36f000ba7f3cf": {
"model_module": "@jupyter-widgets/base",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"217de6953d134b028643fe7278574ec0": {
"model_module": "@jupyter-widgets/controls",
"model_name": "IntProgressModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "IntProgressModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "ProgressView",
"bar_style": "success",
"description": "100%",
"description_tooltip": null,
"layout": "IPY_MODEL_ab137a3ff493410cb8368d20018a35b7",
"max": 1500,
"min": 0,
"orientation": "horizontal",
"style": "IPY_MODEL_af84749aacfb4d0bb42fa23df2c9ae5d",
"value": 1500
}
},
"30b59bbc8c9c46bdb03207ea0270e086": {
"model_module": "@jupyter-widgets/controls",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_865a4c3e5c744e928835f5ae5bfbc760",
"placeholder": "​",
"style": "IPY_MODEL_df2736cc330046ba9d461c3d8c1fc00e",
"value": " 1500/1500 [02:08&lt;00:00, 11.70it/s]"
}
},
"4e0fff7fd7f74c5e8e7f3b7b74ce1daf": {
"model_module": "@jupyter-widgets/controls",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_8aeaa320d7b1411c89c2856c6d447413",
"placeholder": "​",
"style": "IPY_MODEL_9cb253edac994418a9f463965b82d83d",
"value": " 1500/1500 [05:46&lt;00:00, 4.33it/s]"
}
},
"4f76335fb6bf470282b1e8c2d9138fac": {
"model_module": "@jupyter-widgets/base",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"768ed45dc77245619fc1a71d370e4170": {
"model_module": "@jupyter-widgets/controls",
"model_name": "ProgressStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "ProgressStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"bar_color": null,
"description_width": "initial"
}
},
"7b738450b7d3493489274ca67832372d": {
"model_module": "@jupyter-widgets/controls",
"model_name": "IntProgressModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "IntProgressModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "ProgressView",
"bar_style": "success",
"description": "100%",
"description_tooltip": null,
"layout": "IPY_MODEL_a4aa76d02ea9402aad6299d758cbf58a",
"max": 1500,
"min": 0,
"orientation": "horizontal",
"style": "IPY_MODEL_878a49228de84b2a887c5594de48de29",
"value": 1500
}
},
"822929214e8c413bb39934dea80d6f11": {
"model_module": "@jupyter-widgets/controls",
"model_name": "HBoxModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HBoxModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HBoxView",
"box_style": "",
"children": [
"IPY_MODEL_ddd431b7923647d3bcc53c90c56a2337",
"IPY_MODEL_4e0fff7fd7f74c5e8e7f3b7b74ce1daf"
],
"layout": "IPY_MODEL_a9d61d1b1dfb49d08996d1ef19d4d151"
}
},
"85f2503489d24c2297c3589a806c89c1": {
"model_module": "@jupyter-widgets/base",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"865a4c3e5c744e928835f5ae5bfbc760": {
"model_module": "@jupyter-widgets/base",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"878a49228de84b2a887c5594de48de29": {
"model_module": "@jupyter-widgets/controls",
"model_name": "ProgressStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "ProgressStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"bar_color": null,
"description_width": "initial"
}
},
"8aeaa320d7b1411c89c2856c6d447413": {
"model_module": "@jupyter-widgets/base",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"9885433b7d4f407582abc288b9dabfdf": {
"model_module": "@jupyter-widgets/controls",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_fd8bebdc35054ca285a2ecd0e6a212a5",
"placeholder": "​",
"style": "IPY_MODEL_16d85c7b33f1437a965dbab57d3d5c79",
"value": " 1500/1500 [05:42&lt;00:00, 4.38it/s]"
}
},
"9cb253edac994418a9f463965b82d83d": {
"model_module": "@jupyter-widgets/controls",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"a4aa76d02ea9402aad6299d758cbf58a": {
"model_module": "@jupyter-widgets/base",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"a863c0ca60ec48c5a4cdd57bc91c0e84": {
"model_module": "@jupyter-widgets/base",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"a9d61d1b1dfb49d08996d1ef19d4d151": {
"model_module": "@jupyter-widgets/base",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"ab137a3ff493410cb8368d20018a35b7": {
"model_module": "@jupyter-widgets/base",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"af84749aacfb4d0bb42fa23df2c9ae5d": {
"model_module": "@jupyter-widgets/controls",
"model_name": "ProgressStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "ProgressStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"bar_color": null,
"description_width": "initial"
}
},
"be5c0e141898481fabcbca8a251de690": {
"model_module": "@jupyter-widgets/controls",
"model_name": "HBoxModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HBoxModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HBoxView",
"box_style": "",
"children": [
"IPY_MODEL_7b738450b7d3493489274ca67832372d",
"IPY_MODEL_fda4fe65884c40a59d32dd8b97fadb74"
],
"layout": "IPY_MODEL_206a29154ce846568cc36f000ba7f3cf"
}
},
"d10b8192bfda40b790a8e547ab4c0ea7": {
"model_module": "@jupyter-widgets/base",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"da9acea59c634fd7b92fbac5c5ccef17": {
"model_module": "@jupyter-widgets/controls",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"ddd431b7923647d3bcc53c90c56a2337": {
"model_module": "@jupyter-widgets/controls",
"model_name": "IntProgressModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "IntProgressModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "ProgressView",
"bar_style": "success",
"description": "100%",
"description_tooltip": null,
"layout": "IPY_MODEL_4f76335fb6bf470282b1e8c2d9138fac",
"max": 1500,
"min": 0,
"orientation": "horizontal",
"style": "IPY_MODEL_768ed45dc77245619fc1a71d370e4170",
"value": 1500
}
},
"df2736cc330046ba9d461c3d8c1fc00e": {
"model_module": "@jupyter-widgets/controls",
"model_name": "DescriptionStyleModel",
"state": {
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "DescriptionStyleModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "StyleView",
"description_width": ""
}
},
"dfd11ed9a4bc469ca95a22c0fcad7d05": {
"model_module": "@jupyter-widgets/controls",
"model_name": "IntProgressModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "IntProgressModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "ProgressView",
"bar_style": "success",
"description": "100%",
"description_tooltip": null,
"layout": "IPY_MODEL_a863c0ca60ec48c5a4cdd57bc91c0e84",
"max": 1500,
"min": 0,
"orientation": "horizontal",
"style": "IPY_MODEL_0b4166703dc6439bb2534e23e3ab9921",
"value": 1500
}
},
"e3daf6ee9be947c385914a74cb51bac9": {
"model_module": "@jupyter-widgets/base",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"fd8bebdc35054ca285a2ecd0e6a212a5": {
"model_module": "@jupyter-widgets/base",
"model_name": "LayoutModel",
"state": {
"_model_module": "@jupyter-widgets/base",
"_model_module_version": "1.2.0",
"_model_name": "LayoutModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/base",
"_view_module_version": "1.2.0",
"_view_name": "LayoutView",
"align_content": null,
"align_items": null,
"align_self": null,
"border": null,
"bottom": null,
"display": null,
"flex": null,
"flex_flow": null,
"grid_area": null,
"grid_auto_columns": null,
"grid_auto_flow": null,
"grid_auto_rows": null,
"grid_column": null,
"grid_gap": null,
"grid_row": null,
"grid_template_areas": null,
"grid_template_columns": null,
"grid_template_rows": null,
"height": null,
"justify_content": null,
"justify_items": null,
"left": null,
"margin": null,
"max_height": null,
"max_width": null,
"min_height": null,
"min_width": null,
"object_fit": null,
"object_position": null,
"order": null,
"overflow": null,
"overflow_x": null,
"overflow_y": null,
"padding": null,
"right": null,
"top": null,
"visibility": null,
"width": null
}
},
"fda4fe65884c40a59d32dd8b97fadb74": {
"model_module": "@jupyter-widgets/controls",
"model_name": "HTMLModel",
"state": {
"_dom_classes": [],
"_model_module": "@jupyter-widgets/controls",
"_model_module_version": "1.5.0",
"_model_name": "HTMLModel",
"_view_count": null,
"_view_module": "@jupyter-widgets/controls",
"_view_module_version": "1.5.0",
"_view_name": "HTMLView",
"description": "",
"description_tooltip": null,
"layout": "IPY_MODEL_d10b8192bfda40b790a8e547ab4c0ea7",
"placeholder": "​",
"style": "IPY_MODEL_da9acea59c634fd7b92fbac5c5ccef17",
"value": " 1500/1500 [00:03&lt;00:00, 423.17it/s]"
}
}
}
}
},
"nbformat": 4,
"nbformat_minor": 1
}
Reference in New Issue View Git Blame Copy Permalink