{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "2af2050c",
      "metadata": {
        "id": "2af2050c"
      },
      "source": [
        "# **Tensorflow 簡介**\n",
        "Tensorflow 是一個機器學習的開發平台，提供使用者實現 MLP, CNN, RNN 等等的深度學習演算法，以下會介紹 Tensorflow 達到深度學習演算法所需的基本概念。\n",
        "\n",
        "## 本章節內容大綱\n",
        "* ### [建構支援數值計算的高維度矩陣（Tensor, Multidimensional-array）](#Tensor,Multidimensional-array)\n",
        "* ### [自動計算微分值（Automatic differentiation）](#AutomaticDifferentiation)\n",
        "* ### [模型建置以及訓練（Model construction, training）](#ModelConstruction,training)\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "84eb7d44",
      "metadata": {
        "id": "84eb7d44"
      },
      "outputs": [],
      "source": [
        "# 匯入套件\n",
        "import tensorflow as tf\n",
        "import numpy as np"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3fd8260c",
      "metadata": {
        "id": "3fd8260c"
      },
      "source": [
        "<a name=\"Tensor,Multidimensional-array\"></a>\n",
        "## 建構支援數值計算的高維度矩陣（Tensor, Multidimensional-array）\n",
        "Tensorflow 張量（Tensor），寫法與 numpy 陣列類似，也能任意從 scalar，list，或 ndarray 的型態轉換成 Tensor"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7a3511ef",
      "metadata": {
        "id": "7a3511ef"
      },
      "outputs": [],
      "source": [
        "a = tf.constant([1, 2, 3, 4, 5], dtype='float32')\n",
        "print(a)\n",
        "\n",
        "b = tf.constant(np.array([1, 2, 3, 4, 5]), dtype='float32')\n",
        "print(b)\n",
        "\n",
        "c = tf.convert_to_tensor([1, 2, 3, 4, 5], dtype='float32')\n",
        "print(c)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b2422616",
      "metadata": {
        "id": "b2422616"
      },
      "outputs": [],
      "source": [
        "# tf.Tensor -> ndarray\n",
        "a.numpy()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1ab353a5",
      "metadata": {
        "id": "1ab353a5"
      },
      "source": [
        "* ### Tensor 性質"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "68ccce48",
      "metadata": {
        "id": "68ccce48"
      },
      "outputs": [],
      "source": [
        "t = tf.random.normal((1, 2, 4))\n",
        "\n",
        "print('數據類型：', t.dtype)   # 數據類型\n",
        "print('形狀：', t.shape)       # 形狀\n",
        "print('維度數：', t.ndim)      # 維度數"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "180ddc7f",
      "metadata": {
        "id": "180ddc7f"
      },
      "source": [
        "* ### Tensor 操作"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "93879ea6",
      "metadata": {
        "id": "93879ea6"
      },
      "source": [
        "* #### slice\n",
        "從 Tensor 中選取部分內容"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e0503983",
      "metadata": {
        "id": "e0503983"
      },
      "outputs": [],
      "source": [
        "t = tf.constant([[0, 1, 2, 3, 4],\n",
        "                 [5, 6, 7, 8, 9],\n",
        "                 [10, 11, 12, 13, 14],\n",
        "                 [15, 16, 17, 18, 19]])\n",
        "\n",
        "print(t[1:3, 2:])\n",
        "print(tf.slice(t, begin=[1, 2], size=[2, 3]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f0d43df4",
      "metadata": {
        "id": "f0d43df4"
      },
      "outputs": [],
      "source": [
        "t = tf.constant([[[1, 3, 5, 7],\n",
        "                  [9, 11, 13, 15]],\n",
        "                 [[17, 19, 21, 23],\n",
        "                  [25, 27, 29, 31]]])\n",
        "\n",
        "print(t[1:2, 1:2, 0:2])\n",
        "print(tf.slice(t, begin=[1, 1, 0], size=[1, 1, 2]))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7e6aa644",
      "metadata": {
        "id": "7e6aa644"
      },
      "source": [
        "* #### gather\n",
        "從 Tensor 中依據＂軸向＂（axis）以及＂索引＂（index）選取部分內容"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b85bd541",
      "metadata": {
        "id": "b85bd541"
      },
      "outputs": [],
      "source": [
        "t = tf.constant([0, 1, 2, 3, 4, 5, 6, 7])\n",
        "\n",
        "print(tf.gather(t,\n",
        "                indices=[0, 3, 6]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "700336b2",
      "metadata": {
        "id": "700336b2"
      },
      "outputs": [],
      "source": [
        "t = tf.constant([[0, 5],\n",
        "                 [1, 6],\n",
        "                 [2, 7],\n",
        "                 [3, 8],\n",
        "                 [4, 9]])\n",
        "\n",
        "print(tf.gather_nd(t,\n",
        "                   indices=[[2], [3], [0]]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ff6d62b1",
      "metadata": {
        "id": "ff6d62b1"
      },
      "outputs": [],
      "source": [
        "t = tf.constant([[[0, 1, 2],\n",
        "                  [3, 4, 5],\n",
        "                  [6, 7, 8]],\n",
        "                 [[9, 10, 11],\n",
        "                  [12, 13, 14],\n",
        "                  [15, 16, 17]]])\n",
        "\n",
        "print(tf.gather_nd(t,\n",
        "                   indices=[[0, 0], [0, 2], [1, 0], [1, 2]]))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "818faf03",
      "metadata": {
        "id": "818faf03"
      },
      "source": [
        "* #### reshape\n",
        "將 Tensor 改變成指定的形狀"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f89b1dc2",
      "metadata": {
        "id": "f89b1dc2"
      },
      "outputs": [],
      "source": [
        "t = [[1, 2, 3],\n",
        "     [4, 5, 6]]\n",
        "print(tf.reshape(t, shape=[6]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "67bebf3e",
      "metadata": {
        "id": "67bebf3e"
      },
      "outputs": [],
      "source": [
        "t = [[1, 2, 3],\n",
        "     [4, 5, 6]]\n",
        "\n",
        "print(tf.reshape(t, shape=[3, 2]))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2fb3b2ec",
      "metadata": {
        "id": "2fb3b2ec"
      },
      "source": [
        "* #### expand_dims\n",
        "增加 Tensor 維度"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "35314323",
      "metadata": {
        "id": "35314323"
      },
      "outputs": [],
      "source": [
        "t = tf.random.normal((2, 2, 3))\n",
        "\n",
        "print(tf.expand_dims(t, axis=0))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "51e0a497",
      "metadata": {
        "id": "51e0a497"
      },
      "outputs": [],
      "source": [
        "t = tf.random.normal(shape=(2, 2, 3))\n",
        "\n",
        "print(tf.expand_dims(t, axis=-1))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4a46d6f1",
      "metadata": {
        "id": "4a46d6f1"
      },
      "source": [
        "* #### squeeze\n",
        "壓縮 Tensor 維度"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "42a53412",
      "metadata": {
        "id": "42a53412"
      },
      "outputs": [],
      "source": [
        "t = tf.random.normal((2, 2, 1))\n",
        "\n",
        "print(tf.squeeze(t, axis=-1))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "3e0f5741",
      "metadata": {
        "id": "3e0f5741"
      },
      "outputs": [],
      "source": [
        "t = tf.random.normal((2, 1, 3, 1))\n",
        "\n",
        "print(tf.squeeze(t, axis=[1, 3]))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ab557ad2",
      "metadata": {
        "id": "ab557ad2"
      },
      "source": [
        "* #### transpose\n",
        "轉置 Tensor"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bf8fcdde",
      "metadata": {
        "id": "bf8fcdde"
      },
      "outputs": [],
      "source": [
        "t = [[1, 2, 3],\n",
        "     [4, 5, 6]]\n",
        "\n",
        "tf.transpose(t, perm=[1, 0])"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4c116339",
      "metadata": {
        "id": "4c116339"
      },
      "source": [
        "* #### math\n",
        "數學計算，包括加乘除"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "45ebf80a",
      "metadata": {
        "id": "45ebf80a"
      },
      "outputs": [],
      "source": [
        "a = tf.constant([[1, 2],\n",
        "                 [3, 4]])\n",
        "b = tf.constant([[1, 1],\n",
        "                 [1, 1]])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ee8dd8b7",
      "metadata": {
        "id": "ee8dd8b7"
      },
      "outputs": [],
      "source": [
        "print(a + b, '\\n')  # element-wise addition\n",
        "print(a - b, '\\n')  # element-wise subtraction\n",
        "print(a * b, '\\n')  # element-wise multiplication\n",
        "print(a @ b, '\\n')  # matrix multiplication"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a5c9facb",
      "metadata": {
        "id": "a5c9facb"
      },
      "outputs": [],
      "source": [
        "print(tf.add(a, b), \"\\n\")       # element-wise addition\n",
        "print(tf.subtract(a, b), '\\n')  # element-wise subtraction\n",
        "print(tf.multiply(a, b), \"\\n\")  # element-wise multiplication\n",
        "print(tf.matmul(a, b), \"\\n\")    # matrix multiplication"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "9a7827e9",
      "metadata": {
        "id": "9a7827e9"
      },
      "outputs": [],
      "source": [
        "c = tf.constant([[4.0, 5.0], [10.0, 1.0]])\n",
        "\n",
        "# Find the largest value\n",
        "print(tf.reduce_max(c))\n",
        "# Compute the average value\n",
        "print(tf.reduce_mean(c))\n",
        "# Find the index of the largest value\n",
        "print(tf.math.argmax(c))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cf1db39a",
      "metadata": {
        "id": "cf1db39a"
      },
      "source": [
        "-------------"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "79af21df",
      "metadata": {
        "id": "79af21df"
      },
      "source": [
        "Tensorflow 張量與 Numpy 陣列不同的是，又可分為兩種\n",
        "* 無法指派新的值的型態 Tensor\n",
        "* 可以指派新的值的型態 Variable"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "0962c003",
      "metadata": {
        "id": "0962c003"
      },
      "outputs": [],
      "source": [
        "# numpy 陣列\n",
        "a = np.ones((1, 2))\n",
        "print(a)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "70f40d76",
      "metadata": {
        "id": "70f40d76"
      },
      "outputs": [],
      "source": [
        "# Numpy 陣列可接受指派新的值\n",
        "a[0, 0] = 0\n",
        "print(a)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8772b5a4",
      "metadata": {
        "id": "8772b5a4"
      },
      "outputs": [],
      "source": [
        "# Tensorflow 張量\n",
        "b = tf.ones((1, 2))\n",
        "print(b)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1462b82a",
      "metadata": {
        "id": "1462b82a"
      },
      "outputs": [],
      "source": [
        "# Tensorflow 張量不接受指派新的值（以下程式碼會發生錯誤）\n",
        "b[0, 0] = 0\n",
        "print(b)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "da509938",
      "metadata": {
        "id": "da509938"
      },
      "outputs": [],
      "source": [
        "# Tensorflow 僅能使用 Variable 指派新的值\n",
        "v = tf.Variable(tf.ones((1, 2)))\n",
        "print(v)\n",
        "\n",
        "v.assign(tf.zeros((1, 2)))\n",
        "print(v)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a0b06662",
      "metadata": {
        "id": "a0b06662"
      },
      "source": [
        "在類神經網絡中，可以訓練的變數都會以 Variable 的形式存在，才能更新數值。\n",
        "\n",
        "------------------"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cd838b5c",
      "metadata": {
        "id": "cd838b5c"
      },
      "source": [
        "<a name=\"AutomaticDifferentiation\"></a>\n",
        "## 自動計算微分值（Automatic Differentiation）\n",
        "在深度學習演算法當中，很重要的部分就是如何做模型的更新，其中牽涉到對變數做偏微分。"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8a39cc47",
      "metadata": {
        "id": "8a39cc47"
      },
      "source": [
        ">若此函數 $f(x) = x^2+3x-5$ 對 $x$ 做偏微分，則能得到 $f^\\prime(x) = 2x+3$\n",
        ">\n",
        ">將 $x = 1$ 代入函數，得到 $f(x)=-1$，$f^\\prime(x) = 5$\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f9961acd",
      "metadata": {
        "id": "f9961acd"
      },
      "outputs": [],
      "source": [
        "def f(x):\n",
        "    y = x ** 2 + 3 * x - 5\n",
        "    return y\n",
        "\n",
        "\n",
        "x = tf.Variable(1.0)\n",
        "f(x)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b089c994",
      "metadata": {
        "id": "b089c994"
      },
      "outputs": [],
      "source": [
        "with tf.GradientTape() as tape:\n",
        "    y = f(x)\n",
        "g_x = tape.gradient(y, x)  # g(x) = f'(x) = dy/dx\n",
        "\n",
        "g_x"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0d631bea",
      "metadata": {
        "id": "0d631bea"
      },
      "source": [
        "<a name=\"ModelConstruction,training\"></a>\n",
        "## 模型建置以及訓練（Model Construction, Training）"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "31a9c8a6",
      "metadata": {
        "id": "31a9c8a6"
      },
      "source": [
        "![](https://hackmd.io/_uploads/HyQkOtU-a.png)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "24990f5c",
      "metadata": {
        "id": "24990f5c"
      },
      "source": [
        "* ### 適合新手：\n",
        "\n",
        "以 tf.keras 的方式建構模型以及訓練模型，能夠更快速的完成整個機器學習過程，將會在 Part2，Part3 接續課程中介紹。\n",
        "* ### 適合專家：\n",
        "\n",
        "可以更客製化的建置模型、訓練過程等等，於 Part4 做介紹。（在本課程中將會列為選修內容）"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a4554977",
      "metadata": {
        "id": "a4554977"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.12"
    },
    "colab": {
      "provenance": []
    },
    "accelerator": "GPU",
    "gpuClass": "standard"
  },
  "nbformat": 4,
  "nbformat_minor": 5
}