{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "2af2050c",
      "metadata": {
        "id": "2af2050c"
      },
      "source": [
        "# **PyTorch 簡介**\n",
        "PyTorch 是一個機器學習的開發平台，提供使用者實現 MLP, CNN, RNN 等等的深度學習演模型與演算法，以下會介紹 PyTorch 達到深度學習演算法所需的基本概念。\n",
        "\n",
        "## 本章節內容大綱\n",
        "* ### [建構支援數值計算的高維度矩陣（Tensor, Multidimensional-array）](#Tensor,Multidimensional-array)\n",
        "* ### [自動計算微分值（Automatic differentiation）](#AutomaticDifferentiation)\n",
        "* ### [模型建置以及訓練（Model construction, training）](#ModelConstruction,training)\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "84eb7d44",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "84eb7d44",
        "outputId": "c9129955-2eec-4ad3-cfea-5f85e4610d27"
      },
      "outputs": [],
      "source": [
        "# 匯入套件\n",
        "import torch\n",
        "import numpy as np\n",
        "\n",
        "print(torch.__version__)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3fd8260c",
      "metadata": {
        "id": "3fd8260c"
      },
      "source": [
        "<a name=\"Tensor,Multidimensional-array\"></a>\n",
        "## 建構支援數值計算的高維度矩陣（Tensor, Multidimensional-array）\n",
        "PyTorch 張量（torch.Tensor），寫法與 numpy 陣列類似，也能任意從 scalar，list，或 np.ndarray 的型態轉換成 torch.Tensor"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7a3511ef",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7a3511ef",
        "outputId": "ed30c52c-e94d-4ade-9c76-1f2ed6951284"
      },
      "outputs": [],
      "source": [
        "a = torch.tensor([1, 2, 3, 4, 5], dtype=torch.float32)\n",
        "print(a)\n",
        "\n",
        "b = torch.tensor(np.array([1, 2, 3, 4, 5]), dtype=torch.float32)\n",
        "print(b)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b2422616",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b2422616",
        "outputId": "88468dac-1ab5-4e2f-a2f2-b1b22b044840"
      },
      "outputs": [],
      "source": [
        "# torch.Tensor -> np.ndarray\n",
        "a.numpy()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1ab353a5",
      "metadata": {
        "id": "1ab353a5"
      },
      "source": [
        "* ### Tensor 性質"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "68ccce48",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "68ccce48",
        "outputId": "dd524eb1-90ce-4bc0-cc90-95953938b57c"
      },
      "outputs": [],
      "source": [
        "t = torch.rand(1, 2, 4)\n",
        "\n",
        "print('數據類型：', t.dtype)   # 數據類型\n",
        "print('形狀：', t.shape)       # 形狀\n",
        "print('維度數：', t.ndim)      # 維度數"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "180ddc7f",
      "metadata": {
        "id": "180ddc7f"
      },
      "source": [
        "* ### Tensor 操作"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "93879ea6",
      "metadata": {
        "id": "93879ea6"
      },
      "source": [
        "* #### slice\n",
        "從 Tensor 中選取部分內容"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e0503983",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "e0503983",
        "outputId": "caf2d727-1c2b-4040-b32f-f998af13ee31"
      },
      "outputs": [],
      "source": [
        "t = torch.tensor([[0, 1, 2, 3, 4],\n",
        "                  [5, 6, 7, 8, 9],\n",
        "                  [10, 11, 12, 13, 14],\n",
        "                  [15, 16, 17, 18, 19]])\n",
        "\n",
        "print(t[1:3, 2:])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f0d43df4",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f0d43df4",
        "outputId": "e9d961f8-4486-4208-ce55-7086f4155434"
      },
      "outputs": [],
      "source": [
        "t = torch.tensor([[[1, 3, 5, 7],\n",
        "                  [9, 11, 13, 15]],\n",
        "                 [[17, 19, 21, 23],\n",
        "                  [25, 27, 29, 31]]])\n",
        "\n",
        "print(t[1:2, 1:2, 0:2])"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "818faf03",
      "metadata": {
        "id": "818faf03"
      },
      "source": [
        "* #### reshape\n",
        "將 Tensor 改變成指定的形狀"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f89b1dc2",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f89b1dc2",
        "outputId": "40e623fc-5df5-4036-dabf-76e5c7f7ed18"
      },
      "outputs": [],
      "source": [
        "t = torch.tensor(\n",
        "    [[1, 2, 3],\n",
        "     [4, 5, 6]]\n",
        ")\n",
        "print(torch.reshape(t, (6,)))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "67bebf3e",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "67bebf3e",
        "outputId": "5e9ce1e6-dfc1-49a5-8054-4e3ead2a6c2b"
      },
      "outputs": [],
      "source": [
        "t = torch.tensor(\n",
        "    [[1, 2, 3],\n",
        "     [4, 5, 6]]\n",
        ")\n",
        "print(torch.reshape(t, (3, 2)))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2fb3b2ec",
      "metadata": {
        "id": "2fb3b2ec"
      },
      "source": [
        "* #### unsqueeze\n",
        "增加 Tensor 維度"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "35314323",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "35314323",
        "outputId": "d6a29560-075f-4f8b-f02a-f2d50dabe011"
      },
      "outputs": [],
      "source": [
        "t = torch.randn((2, 2, 3))\n",
        "\n",
        "print(torch.unsqueeze(t, 0).shape)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "51e0a497",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "51e0a497",
        "outputId": "111ed067-9947-4f5f-9154-de6e8a1e43e7"
      },
      "outputs": [],
      "source": [
        "t = torch.randn(2, 2, 3)\n",
        "\n",
        "print(torch.unsqueeze(t, -1).shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4a46d6f1",
      "metadata": {
        "id": "4a46d6f1"
      },
      "source": [
        "* #### squeeze\n",
        "壓縮 Tensor 維度"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "42a53412",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "42a53412",
        "outputId": "5b9cc1db-3a4a-4df2-ff2b-ffc87ba0b211"
      },
      "outputs": [],
      "source": [
        "t = torch.randn((2, 2, 1))\n",
        "\n",
        "print(torch.squeeze(t, dim=-1).shape)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "3e0f5741",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3e0f5741",
        "outputId": "d0513fca-050b-4501-a436-e8d665c89b2e"
      },
      "outputs": [],
      "source": [
        "t = torch.randn((2, 1, 3, 1))\n",
        "\n",
        "print(torch.squeeze(t, dim=(1, 3)).shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ab557ad2",
      "metadata": {
        "id": "ab557ad2"
      },
      "source": [
        "* #### transpose\n",
        "轉置 Tensor"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bf8fcdde",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bf8fcdde",
        "outputId": "a6a3b50f-f08a-4d44-a598-ed85c344b029"
      },
      "outputs": [],
      "source": [
        "t = torch.tensor(\n",
        "    [[1, 2, 3],\n",
        "     [4, 5, 6]]\n",
        ")\n",
        "torch.transpose(t, 0, 1)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4c116339",
      "metadata": {
        "id": "4c116339"
      },
      "source": [
        "* #### math\n",
        "數學計算，包括加乘除"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "45ebf80a",
      "metadata": {
        "id": "45ebf80a"
      },
      "outputs": [],
      "source": [
        "a = torch.tensor([[1, 2],\n",
        "                  [3, 4]])\n",
        "b = torch.tensor([[1, 1],\n",
        "                  [1, 1]])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ee8dd8b7",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ee8dd8b7",
        "outputId": "5d5082de-6145-45da-e56d-0b9878a2e9a0"
      },
      "outputs": [],
      "source": [
        "print(a + b, '\\n')  # element-wise addition\n",
        "print(a - b, '\\n')  # element-wise subtraction\n",
        "print(a * b, '\\n')  # element-wise multiplication\n",
        "print(a @ b, '\\n')  # matrix multiplication"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a5c9facb",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a5c9facb",
        "outputId": "deca2d1b-f109-4e5d-ad26-f3bfc1f268f3"
      },
      "outputs": [],
      "source": [
        "print(a + b, \"\\n\")        # element-wise addition\n",
        "print(a - b, '\\n')        # element-wise subtraction\n",
        "print(a * b, \"\\n\")        # element-wise multiplication\n",
        "print(torch.matmul(a, b), \"\\n\")  # matrix multiplication"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "9a7827e9",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9a7827e9",
        "outputId": "29df3593-6af9-4e98-8c02-40ee8215b3f0"
      },
      "outputs": [],
      "source": [
        "c = torch.tensor([[4.0, 5.0], [10.0, 1.0]])\n",
        "\n",
        "# Find the largest value\n",
        "print(torch.max(c))\n",
        "# Compute the average value\n",
        "print(torch.mean(c))\n",
        "# Find the index of the largest value\n",
        "print(torch.argmax(c))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cf1db39a",
      "metadata": {
        "id": "cf1db39a"
      },
      "source": [
        "-------------"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cd838b5c",
      "metadata": {
        "id": "cd838b5c"
      },
      "source": [
        "<a name=\"AutomaticDifferentiation\"></a>\n",
        "## 自動計算微分值（Automatic Differentiation）\n",
        "在深度學習演算法當中，很重要的部分就是如何做模型的更新，其中牽涉到對變數做偏微分。"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8a39cc47",
      "metadata": {
        "id": "8a39cc47"
      },
      "source": [
        ">若此函數 $f(x) = x^2+3x-5$ 對 $x$ 做偏微分，則能得到 $f^\\prime(x) = 2x+3$\n",
        ">\n",
        ">將 $x = 1$ 代入函數，得到 $f(x)=-1$，$f^\\prime(x) = 5$\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f9961acd",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f9961acd",
        "outputId": "d346181b-3886-48c4-8ff1-39850a0b675d"
      },
      "outputs": [],
      "source": [
        "def f(x):\n",
        "    y = x ** 2 + 3 * x - 5\n",
        "    return y\n",
        "\n",
        "\n",
        "x = torch.tensor(1.0, requires_grad=True)\n",
        "print(f(x))\n",
        "\n",
        "y = f(x)\n",
        "y.backward()  # 反向傳播計算梯度\n",
        "\n",
        "g_x = x.grad  # g(x) = f'(x) = dy/dx\n",
        "print(g_x)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0d631bea",
      "metadata": {
        "id": "0d631bea"
      },
      "source": [
        "<a name=\"ModelConstruction,training\"></a>\n",
        "## 模型建置以及訓練（Model Construction, Training）"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "24990f5c",
      "metadata": {
        "id": "24990f5c"
      },
      "source": [
        "* ### 適合新手：\n",
        "\n",
        "以 toch.nn.Sequential 的方式建構模型以及訓練模型，能夠更快速的建立簡單模型，將會在 Part2，Part3 接續課程中介紹。\n",
        "* ### 適合專家：\n",
        "\n",
        "繼承torch.nn.Module 可以更客製化的建置模型、訓練過程等等，於 Part4 做介紹。（在本課程中將會列為選修內容）"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a4554977",
      "metadata": {
        "id": "a4554977"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.12"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}