{
"cells": [
{
"cell_type": "markdown",
"id": "e51223b0",
"metadata": {
"id": "e51223b0"
},
"source": [
"# **模型調校（Model Tuning）**\n",
"此份程式碼會提供針對某資料集的模型調校策略，以及比較其超參數的選擇。\n",
"\n",
"## 本章節內容大綱\n",
"* ### [損失函數（Loss function）](#LossFunction)\n",
"* ### [激活函數（Activation function）](#ActivationFunction)\n",
"* ### [優化器（Optimizer）](#Optimizer)\n",
"* ### [學習率（Learning rate）](#LearningRate)\n",
"* ### [模型架構（Model architecture）](#ModelArchitecture)"
]
},
{
"cell_type": "markdown",
"id": "33255f75",
"metadata": {
"id": "33255f75"
},
"source": [
"## 匯入套件"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bac12f0a",
"metadata": {
"id": "bac12f0a"
},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"\n",
"# Tensorflow 相關套件\n",
"import tensorflow as tf\n",
"from tensorflow import keras\n",
"from tensorflow.keras import layers"
]
},
{
"cell_type": "markdown",
"id": "495c8b24",
"metadata": {
"id": "495c8b24"
},
"source": [
"## 創建資料集／載入資料集（Dataset Creating / Loading）"
]
},
{
"cell_type": "code",
"source": [
"# 上傳資料\n",
"!wget -q https://github.com/TA-aiacademy/course_3.0/releases/download/DL/Data_part3.zip\n",
"!unzip -q Data_part3.zip"
],
"metadata": {
"id": "Ny4jdxCvv7XU"
},
"id": "Ny4jdxCvv7XU",
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "899342e7",
"metadata": {
"id": "899342e7"
},
"outputs": [],
"source": [
"train_df = pd.read_csv('./Data/News_train.csv')\n",
"test_df = pd.read_csv('./Data/News_test.csv')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8be123e5",
"metadata": {
"id": "8be123e5"
},
"outputs": [],
"source": [
"train_df.head()"
]
},
{
"cell_type": "markdown",
"id": "e37d2723",
"metadata": {
"id": "e37d2723"
},
"source": [
"* #### 新聞文章資料集\n",
"訓練集，測試集分別為 7728，1907 筆，4081 種常用字詞，若在同一篇新聞中出現該字詞為 1，若否則為 0，y_category 標記文章類別，共 11 種類別。"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7aace4ec",
"metadata": {
"id": "7aace4ec"
},
"outputs": [],
"source": [
"X_df = train_df.iloc[:, :-1].values\n",
"y_df = train_df.y_category.values"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2fdea65d",
"metadata": {
"id": "2fdea65d"
},
"outputs": [],
"source": [
"X_test = test_df.iloc[:, :-1].values\n",
"y_test = test_df.y_category.values"
]
},
{
"cell_type": "markdown",
"id": "a5ca0f5f",
"metadata": {
"id": "a5ca0f5f"
},
"source": [
"## 資料前處理（Data Preprocessing）"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2ead5bdb",
"metadata": {
"id": "2ead5bdb"
},
"outputs": [],
"source": [
"from sklearn.preprocessing import StandardScaler, MinMaxScaler\n",
"# Feature scaling\n",
"sc = StandardScaler()\n",
"X_scale = sc.fit_transform(X_df, y_df)\n",
"X_test_scale = sc.transform(X_test)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9de9f563",
"metadata": {
"id": "9de9f563"
},
"outputs": [],
"source": [
"# Convert to One-Hot encoding\n",
"y_onehot = keras.utils.to_categorical(y_df)\n",
"y_test_onehot = keras.utils.to_categorical(y_test)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b00c5e3a",
"metadata": {
"id": "b00c5e3a"
},
"outputs": [],
"source": [
"# train, valid/test dataset split\n",
"from sklearn.model_selection import train_test_split\n",
"X_train, X_valid, y_train, y_valid = train_test_split(X_scale, y_onehot,\n",
" test_size=0.2,\n",
" random_state=17,\n",
" stratify=y_df)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6be0c87c",
"metadata": {
"id": "6be0c87c"
},
"outputs": [],
"source": [
"print(f'X_train shape: {X_train.shape}')\n",
"print(f'X_valid shape: {X_valid.shape}')\n",
"print(f'y_train shape: {y_train.shape}')\n",
"print(f'y_valid shape: {y_valid.shape}')"
]
},
{
"cell_type": "markdown",
"id": "02cf5671",
"metadata": {
"id": "02cf5671"
},
"source": [
"## 模型建置（Model Building）"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fc6e3e67",
"metadata": {
"id": "fc6e3e67"
},
"outputs": [],
"source": [
"def build_model(input_shape, output_shape):\n",
" keras.backend.clear_session() # 重置 keras 的所有狀態\n",
" tf.random.set_seed(17) # 設定 tensorflow 隨機種子\n",
"\n",
" model = keras.models.Sequential()\n",
" model.add(layers.Dense(16, # 神經元個數\n",
" input_shape=input_shape, # 輸入形狀\n",
" activation='sigmoid')) # 激活函數\n",
" model.add(layers.Dense(16,\n",
" activation='sigmoid'))\n",
" model.add(layers.Dense(output_shape,\n",
" activation='softmax'))\n",
" return model"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5a41671f",
"metadata": {
"id": "5a41671f"
},
"outputs": [],
"source": [
"model = build_model(X_train[0].shape, y_onehot.shape[1])\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"id": "fce8b56e",
"metadata": {
"id": "fce8b56e"
},
"source": [
"## 模型訓練（Model Training）"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "533b4950",
"metadata": {
"id": "533b4950"
},
"outputs": [],
"source": [
"model.compile(optimizer='rmsprop',\n",
" loss='categorical_crossentropy',\n",
" metrics=['acc'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "203d09bc",
"metadata": {
"id": "203d09bc"
},
"outputs": [],
"source": [
"history = model.fit(X_train, y_train,\n",
" epochs=20,\n",
" batch_size=64,\n",
" validation_data=(X_valid, y_valid))"
]
},
{
"cell_type": "markdown",
"id": "48403507",
"metadata": {
"id": "48403507"
},
"source": [
"## 模型評估（Model Evaluation）"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "16a070b5",
"metadata": {
"id": "16a070b5"
},
"outputs": [],
"source": [
"train_loss = history.history['loss']\n",
"train_acc = history.history['acc']\n",
"\n",
"valid_loss = history.history['val_loss']\n",
"valid_acc = history.history['val_acc']"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0b24dabb",
"metadata": {
"id": "0b24dabb"
},
"outputs": [],
"source": [
"plt.figure(figsize=(15, 4))\n",
"plt.subplot(1, 2, 1)\n",
"plt.plot(range(len(train_loss)), train_loss, label='train_loss')\n",
"plt.plot(range(len(valid_loss)), valid_loss, label='valid_loss')\n",
"plt.xlabel('Epochs')\n",
"plt.ylabel('Loss')\n",
"plt.legend()\n",
"\n",
"plt.subplot(1, 2, 2)\n",
"plt.plot(range(len(train_acc)), train_acc, label='train_acc')\n",
"plt.plot(range(len(valid_acc)), valid_acc, label='valid_acc')\n",
"plt.xlabel('Epochs')\n",
"plt.ylabel('Accuracy')\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "98120751",
"metadata": {
"id": "98120751"
},
"source": [
"## 模型調校"
]
},
{
"cell_type": "markdown",
"id": "0316d055",
"metadata": {
"id": "0316d055"
},
"source": [
"\n"
]
},
{
"cell_type": "markdown",
"id": "ed21b1b9",
"metadata": {
"id": "ed21b1b9"
},
"source": [
"\n",
"* ## 損失函數（Loss function）\n",
"tf.keras.losses: https://www.tensorflow.org/api_docs/python/tf/keras/losses"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a4ab32b2",
"metadata": {
"id": "a4ab32b2"
},
"outputs": [],
"source": [
"# 以下放置要比較的 loss function\n",
"loss_funcs = ['mean_squared_error',\n",
" 'categorical_crossentropy',\n",
" 'mean_absolute_error']\n",
"\n",
"batch_size = 64\n",
"epochs = 20\n",
"\n",
"# 建立兩個 list 記錄選用不同 loss function 的訓練結果\n",
"all_loss, all_acc = [], []\n",
"\n",
"# 迭代不同的 loss function 去訓練模型\n",
"for loss_f in loss_funcs:\n",
" print(f'Running model, loss = {loss_f}')\n",
"\n",
" # 確保每次都是訓練新的模型，而不是接續上一輪的模型\n",
" model = build_model(X_train[0].shape, y_onehot.shape[1])\n",
" model.compile(loss=loss_f,\n",
" optimizer='rmsprop',\n",
" metrics=['acc'])\n",
"\n",
" # 確保每次都設定一樣的參數\n",
" history = model.fit(X_train, y_train,\n",
" batch_size=batch_size,\n",
" epochs=epochs,\n",
" verbose=0,\n",
" validation_data=(X_valid, y_valid))\n",
"\n",
" # 將訓練過程記錄下來\n",
" all_loss.append(history.history['loss'])\n",
" all_acc.append(history.history['acc'])\n",
"print('----------------- training done! -----------------')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e5ddd8df",
"metadata": {
"id": "e5ddd8df"
},
"outputs": [],
"source": [
"# 視覺化訓練過程\n",
"plt.figure(figsize=(15, 7))\n",
"\n",
"# 繪製 Training loss\n",
"plt.subplot(121)\n",
"for k in range(len(loss_funcs)):\n",
" plt.plot(range(len(all_loss[k])), all_loss[k], label=loss_funcs[k])\n",
"plt.title('Loss')\n",
"\n",
"# 繪製 Training accuracy\n",
"plt.subplot(122)\n",
"for k in range(len(loss_funcs)):\n",
" plt.plot(range(len(all_acc[k])), all_acc[k], label=loss_funcs[k])\n",
"plt.title('Accuracy')\n",
"\n",
"plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=2.)\n",
"plt.ylim((0, 1))\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "48d4b3d2",
"metadata": {
"id": "48d4b3d2"
},
"source": [
"---\n",
"\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "51977b51",
"metadata": {
"id": "51977b51"
},
"source": [
"\n",
"* ## 激活函數（Activation function）\n",
"tf.keras.activations: https://www.tensorflow.org/api_docs/python/tf/keras/activations"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e0dc4d8c",
"metadata": {
"id": "e0dc4d8c"
},
"outputs": [],
"source": [
"def build_model_activation(input_shape, output_shape, activation):\n",
" # 重新建構一個可以更改 Activation 的模型\n",
" keras.backend.clear_session()\n",
" tf.random.set_seed(17)\n",
"\n",
" model = keras.models.Sequential()\n",
" model.add(layers.Dense(16,\n",
" input_shape=input_shape))\n",
" if activation == 'leakyrelu':\n",
" model.add(layers.LeakyReLU()) # LeakyReLU 位置於 Layers 底下\n",
" else:\n",
" model.add(layers.Activation(activation)) # 由此更改 Activation\n",
"\n",
" model.add(layers.Dense(16))\n",
" if activation == 'leakyrelu':\n",
" model.add(layers.LeakyReLU()) # LeakyReLU 位置於 Layers 底下\n",
" else:\n",
" model.add(layers.Activation(activation)) # 由此更改 Activation\n",
"\n",
" model.add(layers.Dense(output_shape,\n",
" activation='softmax'))\n",
" return model"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e95785db",
"metadata": {
"id": "e95785db"
},
"outputs": [],
"source": [
"# 以下放置要比較的 activation function\n",
"activation_funcs = ['linear',\n",
" 'sigmoid',\n",
" 'tanh',\n",
" 'relu',\n",
" 'softplus',\n",
" 'leakyrelu',\n",
" 'swish']\n",
"\n",
"batch_size = 64\n",
"epochs = 20\n",
"\n",
"# 建立兩個 list 記錄選用不同 activation function 的訓練結果\n",
"all_loss, all_acc = [], []\n",
"\n",
"# 迭代不同的 activation function 去訓練模型\n",
"for activation_f in activation_funcs:\n",
" print(f'Running model, activation = {activation_f}')\n",
"\n",
" # 確保每次都是訓練新的模型，而不是接續上一輪的模型\n",
" model = build_model_activation(X_train[0].shape,\n",
" y_onehot.shape[1],\n",
" activation_f)\n",
" model.compile(loss='categorical_crossentropy',\n",
" optimizer='rmsprop',\n",
" metrics=['acc'])\n",
"\n",
" # 確保每次都設定一樣的參數\n",
" history = model.fit(X_train, y_train,\n",
" batch_size=batch_size,\n",
" epochs=epochs,\n",
" verbose=0,\n",
" validation_data=(X_valid, y_valid))\n",
"\n",
" # 將訓練過程記錄下來\n",
" all_loss.append(history.history['loss'])\n",
" all_acc.append(history.history['acc'])\n",
"print('----------------- training done! -----------------')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b428c947",
"metadata": {
"id": "b428c947"
},
"outputs": [],
"source": [
"# 視覺化訓練過程\n",
"plt.figure(figsize=(15, 7))\n",
"\n",
"# 繪製 Training loss\n",
"plt.subplot(121)\n",
"for k in range(len(activation_funcs)):\n",
" plt.plot(range(len(all_loss[k])), all_loss[k], label=activation_funcs[k])\n",
"plt.title('Loss')\n",
"\n",
"# 繪製 Training accuracy\n",
"plt.subplot(122)\n",
"for k in range(len(activation_funcs)):\n",
" plt.plot(range(len(all_acc[k])), all_acc[k], label=activation_funcs[k])\n",
"plt.title('Accuracy')\n",
"\n",
"plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)\n",
"\n",
"plt.ylim((0, 1))\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6a37ff70",
"metadata": {
"id": "6a37ff70"
},
"outputs": [],
"source": [
"# 視覺化訓練過程\n",
"plt.figure(figsize=(15, 4))\n",
"\n",
"# 繪製 Training loss\n",
"plt.subplot(121)\n",
"for k in range(len(activation_funcs)):\n",
" plt.plot(range(len(all_loss[k])), all_loss[k], label=activation_funcs[k])\n",
"plt.title('Loss')\n",
"plt.ylim((0, 0.3))\n",
"\n",
"# 繪製 Training accuracy\n",
"plt.subplot(122)\n",
"for k in range(len(activation_funcs)):\n",
" plt.plot(range(len(all_acc[k])), all_acc[k], label=activation_funcs[k])\n",
"plt.title('Accuracy')\n",
"\n",
"plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)\n",
"\n",
"plt.ylim((0.95, 0.975))\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "4b3f2f31",
"metadata": {
"id": "4b3f2f31"
},
"source": [
"---\n",
"\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "aa781738",
"metadata": {
"id": "aa781738"
},
"source": [
"\n",
"* ## 優化器（Optimizer）\n",
"tf.keras.optimizers: https://www.tensorflow.org/api_docs/python/tf/keras/optimizers"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0d1d383c",
"metadata": {
"id": "0d1d383c"
},
"outputs": [],
"source": [
"# 以下放置要比較的 optimizer\n",
"optimizer_funcs = ['sgd',\n",
" 'rmsprop',\n",
" 'adam',\n",
" 'nadam']\n",
"\n",
"batch_size = 64\n",
"epochs = 20\n",
"\n",
"# 建立兩個 list 記錄選用不同 optimizer 的訓練結果\n",
"all_loss, all_acc = [], []\n",
"\n",
"# 迭代不同的 optimizer 去訓練模型\n",
"for optimizer_f in optimizer_funcs:\n",
" print(f'Running model, optimizer = {optimizer_f}')\n",
"\n",
" # 確保每次都是訓練新的模型，而不是接續上一輪的模型\n",
" model = build_model_activation(X_train[0].shape,\n",
" y_onehot.shape[1],\n",
" 'tanh')\n",
" model.compile(loss='categorical_crossentropy',\n",
" optimizer=optimizer_f,\n",
" metrics=['acc'])\n",
"\n",
" # 確保每次都設定一樣的參數\n",
" history = model.fit(X_train, y_train,\n",
" batch_size=batch_size,\n",
" epochs=epochs,\n",
" verbose=0,\n",
" validation_data=(X_valid, y_valid))\n",
"\n",
" # 將訓練過程記錄下來\n",
" all_loss.append(history.history['loss'])\n",
" all_acc.append(history.history['acc'])\n",
"print('----------------- training done! -----------------')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5b8f7cf3",
"metadata": {
"id": "5b8f7cf3"
},
"outputs": [],
"source": [
"# 視覺化訓練過程\n",
"plt.figure(figsize=(15, 7))\n",
"\n",
"# 繪製 Training loss\n",
"plt.subplot(121)\n",
"for k in range(len(optimizer_funcs)):\n",
" plt.plot(range(len(all_loss[k])), all_loss[k], label=optimizer_funcs[k])\n",
"plt.title('Loss')\n",
"\n",
"# 繪製 Training accuracy\n",
"plt.subplot(122)\n",
"for k in range(len(optimizer_funcs)):\n",
" plt.plot(range(len(all_acc[k])), all_acc[k], label=optimizer_funcs[k])\n",
"plt.title('Accuracy')\n",
"\n",
"plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=2.)\n",
"plt.ylim((0, 1))\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "21e99f51",
"metadata": {
"id": "21e99f51"
},
"source": [
"\n",
"* ## 學習率（Learning rate）"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f2cff10b",
"metadata": {
"id": "f2cff10b"
},
"outputs": [],
"source": [
"# 以下放置要比較的 learning rate\n",
"lr_list = [0.1, 0.01, 0.001, 0.0001, 0.00001]\n",
"\n",
"batch_size = 64\n",
"epochs = 20\n",
"\n",
"# 建立兩個 list 記錄選用不同 learning rate 的訓練結果\n",
"all_loss, all_acc = [], []\n",
"\n",
"# 迭代不同的 learning rate 去訓練模型\n",
"for lr in lr_list:\n",
" print(f'Running model, learning rate = {lr}')\n",
"\n",
" # 確保每次都是訓練新的模型，而不是接續上一輪的模型\n",
" model = build_model_activation(X_train[0].shape,\n",
" y_onehot.shape[1],\n",
" 'tanh')\n",
" model.compile(loss='categorical_crossentropy',\n",
" optimizer=keras.optimizers.Nadam(lr),\n",
" metrics=['acc'])\n",
"\n",
" # 確保每次都設定一樣的參數\n",
" history = model.fit(X_train, y_train,\n",
" batch_size=batch_size,\n",
" epochs=epochs,\n",
" verbose=0,\n",
" validation_data=(X_valid, y_valid))\n",
"\n",
" # 將訓練過程記錄下來\n",
" all_loss.append(history.history['loss'])\n",
" all_acc.append(history.history['acc'])\n",
"print('----------------- training done! -----------------')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "288def33",
"metadata": {
"id": "288def33"
},
"outputs": [],
"source": [
"# 視覺化訓練過程\n",
"plt.figure(figsize=(15, 7))\n",
"\n",
"# 繪製 Training loss\n",
"plt.subplot(121)\n",
"for k in range(len(lr_list)):\n",
" plt.plot(range(len(all_loss[k])), all_loss[k], label=lr_list[k])\n",
"plt.title('Loss')\n",
"\n",
"# 繪製 Training accuracy\n",
"plt.subplot(122)\n",
"for k in range(len(lr_list)):\n",
" plt.plot(range(len(all_acc[k])), all_acc[k], label=lr_list[k])\n",
"plt.title('Accuracy')\n",
"\n",
"plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=2.)\n",
"plt.ylim((0, 1))\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "615b9c7b",
"metadata": {
"id": "615b9c7b"
},
"source": [
"\n",
"* ## 模型架構（Model architecture）"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5bfb7e23",
"metadata": {
"id": "5bfb7e23"
},
"outputs": [],
"source": [
"def build_model_architecture(input_shape, output_shape, layer, neuron):\n",
" keras.backend.clear_session()\n",
" tf.random.set_seed(17)\n",
"\n",
" model = keras.models.Sequential()\n",
" for i in range(layer):\n",
" model.add(layers.Dense(neuron,\n",
" input_shape=input_shape,\n",
" activation='tanh'))\n",
" model.add(layers.Dense(output_shape,\n",
" activation='softmax'))\n",
" return model"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bb933a0d",
"metadata": {
"id": "bb933a0d"
},
"outputs": [],
"source": [
"# 以下放置要比較的 layers/ neurons\n",
"layers_num = [1, 2, 3]\n",
"neurons_num = [16, 32, 64]\n",
"\n",
"batch_size = 64\n",
"epochs = 20\n",
"\n",
"# 建立兩個 list 記錄選用不同 layers/ neurons 的訓練結果\n",
"all_loss, all_acc = [], []\n",
"\n",
"# 迭代不同的 layers/ neurons 去訓練模型\n",
"for layer in layers_num:\n",
" for neuron in neurons_num:\n",
" print(f'Running model, (layer, neuron) = {(layer, neuron)}')\n",
"\n",
" # 確保每次都是訓練新的模型，而不是接續上一輪的模型\n",
" model = build_model_architecture(X_train[0].shape,\n",
" y_onehot.shape[1],\n",
" layer,\n",
" neuron)\n",
" model.compile(loss='categorical_crossentropy',\n",
" optimizer='nadam',\n",
" metrics=['acc'])\n",
"\n",
" # 確保每次都設定一樣的參數\n",
" history = model.fit(X_train, y_train,\n",
" batch_size=batch_size,\n",
" epochs=epochs,\n",
" verbose=0,\n",
" validation_data=(X_valid, y_valid))\n",
"\n",
" # 將訓練過程記錄下來\n",
" all_loss.append(history.history['loss'])\n",
" all_acc.append(history.history['acc'])\n",
"print('----------------- training done! -----------------')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "74298636",
"metadata": {
"id": "74298636"
},
"outputs": [],
"source": [
"layer_neuron = list(zip(sum([[i]*3 for i in layers_num], []), neurons_num*3))\n",
"\n",
"# 視覺化訓練過程\n",
"plt.figure(figsize=(15, 7))\n",
"\n",
"# 繪製 Training loss\n",
"plt.subplot(121)\n",
"for k in range(len(layer_neuron)):\n",
" plt.plot(range(len(all_loss[k])), all_loss[k], label=layer_neuron[k])\n",
"plt.title('Loss')\n",
"\n",
"# 繪製 Training accuracy\n",
"plt.subplot(122)\n",
"for k in range(len(layer_neuron)):\n",
" plt.plot(range(len(all_acc[k])), all_acc[k], label=layer_neuron[k])\n",
"plt.title('Accuracy')\n",
"\n",
"plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)\n",
"\n",
"plt.ylim((0, 1))\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8e5036b3",
"metadata": {
"id": "8e5036b3"
},
"outputs": [],
"source": [
"# 視覺化訓練過程\n",
"plt.figure(figsize=(15, 4))\n",
"\n",
"# 繪製 Training loss\n",
"plt.subplot(121)\n",
"for k in range(len(layer_neuron)):\n",
" plt.plot(range(len(all_loss[k])), all_loss[k], label=layer_neuron[k])\n",
"plt.title('Loss')\n",
"plt.ylim((0.05, 0.1))\n",
"\n",
"# 繪製 Training accuracy\n",
"plt.subplot(122)\n",
"for k in range(len(layer_neuron)):\n",
" plt.plot(range(len(all_acc[k])), all_acc[k], label=layer_neuron[k])\n",
"plt.title('Accuracy')\n",
"\n",
"plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)\n",
"plt.ylim((0.96, 0.9725))\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "5227388a",
"metadata": {
"id": "5227388a"
},
"source": [
"---\n",
"### Quiz\n",
"請試著利用 Data/pkgo_train.csv 做多元分類問題，預測五個種類的 pokemon，並調整模型（網路層數、神經元數目、激活函數）以及訓練相關的參數得到更高的準確度。"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5a3be942",
"metadata": {
"id": "5a3be942"
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.12"
},
"colab": {
"provenance": []
},
"accelerator": "GPU",
"gpuClass": "standard"
},
"nbformat": 4,
"nbformat_minor": 5
}