107學年度：考慮紋理特徵的高效率複數型類神經網路深度學習架構的研究-II
計畫名稱：考慮紋理特徵的高效率複數型類神經網路深度學習架構的研究-II
執行起迄：2018/08/01~2019/07/31
總核定金額：531,000元
中文摘要：深度學習普遍使用捲積型類神經網路(convolutional neural network，CNN)為基礎，雖然在影像辨識等領域有相當好的表現，但捲積核函數(kernel)的單調性與合理性，使CNN無法適用於所有問題。本研究希望改善因為卷積計算的限制，所造成的特徵擷取受限，並改善深度類神經網路架構的訓練計算效率問題。針對上述議題，本計劃設計了為期兩年的兩階段研究計畫：第一年「紋理特徵自動擷取與選擇的深度學習神經網路」主要工作是設計一種自動擷取與選擇資料集中的特徵紋理，取代卷積計算所擷取的單調的特徵。這部分已經獲得科技部一年期的計畫經費補助---【考慮紋理特徵的高效率複數型類神經網路深度學習架構的研究】（MOST106-2221-E-390-022，108/01～107/07/31）。本年度計畫為整體規劃的第二階段---「多維度多層次複數型類神經網路與ELM形式深度學習架構」。複數值類神經網路(Complex-Valued Neural Network，CxNNs)是一種新型態的神經元架構，以複數為神經元傳遞訊號的表示方式，因為單一複數型態的CxNN神經元就可以達到傳統多個神經元的效果，所以CxNN比傳統倒傳遞類神經網路架構更精簡、具有高效率、高精確度的優點。另外，不同於一般神經網路使用的耗時的倒傳遞迭代學習方式，本計畫發展以極限學習機（Extreme Learning Machines，ELM）為基礎的快速訓練方式，作為深度學習的主要訓練機制。因為ELM考慮訓練誤差的最小平方差，並一次性的求解，其訓練時間往往優於倒傳遞的迭代式訓練，並維持訓練的精確度，應用於多維度、多層次的CxNN架構時，可以更快速的求得結果。
英文摘要：Convolutional neural network (CNN) is the core method of deep learning; however, there are some inherent deficiencies to be solved when CNN is used for practical, usually large-scale, applications. First, features extracted by convolution are specific patterns that are significantly strong, but may be trivial, signals for training recognition models. Secondly, the back-propagation and gradient descent training process is time-consuming and usually inaccurate. A two-stage project is deployed. The first part of this project is to develop texture-based methods for adaptive and robust feature engineering and has been granted by MOST as a one-year research project, “A high-performance deep-learning framework based on texture analysis and complex-valued neural networks (MOST 106-2221-E-390-022), 2017/08/01～2018/07/31.” In the second part, a framework of multi-dimensional multi-layer complex-valued neural networks (CxNNs) is developed. It has been proven that several simple CxNN neurons outperform a bunch of traditional real-valued neurons. The Extreme Learning Machine (ELM) is incorporated with the multi-layer CxNN deep learning framework as the training mechanism. ELM searches for optimized parameters of neurons by fast estimating solutions of the mean square errors of all neurons, instead of fixing errors with iterative gradient descent. With ELM and CxNN, a higher-performance deep learning framework is expected.
執行起迄：2018/08/01~2019/07/31
總核定金額：531,000元
中文摘要：深度學習普遍使用捲積型類神經網路(convolutional neural network，CNN)為基礎，雖然在影像辨識等領域有相當好的表現，但捲積核函數(kernel)的單調性與合理性，使CNN無法適用於所有問題。本研究希望改善因為卷積計算的限制，所造成的特徵擷取受限，並改善深度類神經網路架構的訓練計算效率問題。針對上述議題，本計劃設計了為期兩年的兩階段研究計畫：第一年「紋理特徵自動擷取與選擇的深度學習神經網路」主要工作是設計一種自動擷取與選擇資料集中的特徵紋理，取代卷積計算所擷取的單調的特徵。這部分已經獲得科技部一年期的計畫經費補助---【考慮紋理特徵的高效率複數型類神經網路深度學習架構的研究】（MOST106-2221-E-390-022，108/01～107/07/31）。本年度計畫為整體規劃的第二階段---「多維度多層次複數型類神經網路與ELM形式深度學習架構」。複數值類神經網路(Complex-Valued Neural Network，CxNNs)是一種新型態的神經元架構，以複數為神經元傳遞訊號的表示方式，因為單一複數型態的CxNN神經元就可以達到傳統多個神經元的效果，所以CxNN比傳統倒傳遞類神經網路架構更精簡、具有高效率、高精確度的優點。另外，不同於一般神經網路使用的耗時的倒傳遞迭代學習方式，本計畫發展以極限學習機（Extreme Learning Machines，ELM）為基礎的快速訓練方式，作為深度學習的主要訓練機制。因為ELM考慮訓練誤差的最小平方差，並一次性的求解，其訓練時間往往優於倒傳遞的迭代式訓練，並維持訓練的精確度，應用於多維度、多層次的CxNN架構時，可以更快速的求得結果。
英文摘要：Convolutional neural network (CNN) is the core method of deep learning; however, there are some inherent deficiencies to be solved when CNN is used for practical, usually large-scale, applications. First, features extracted by convolution are specific patterns that are significantly strong, but may be trivial, signals for training recognition models. Secondly, the back-propagation and gradient descent training process is time-consuming and usually inaccurate. A two-stage project is deployed. The first part of this project is to develop texture-based methods for adaptive and robust feature engineering and has been granted by MOST as a one-year research project, “A high-performance deep-learning framework based on texture analysis and complex-valued neural networks (MOST 106-2221-E-390-022), 2017/08/01～2018/07/31.” In the second part, a framework of multi-dimensional multi-layer complex-valued neural networks (CxNNs) is developed. It has been proven that several simple CxNN neurons outperform a bunch of traditional real-valued neurons. The Extreme Learning Machine (ELM) is incorporated with the multi-layer CxNN deep learning framework as the training mechanism. ELM searches for optimized parameters of neurons by fast estimating solutions of the mean square errors of all neurons, instead of fixing errors with iterative gradient descent. With ELM and CxNN, a higher-performance deep learning framework is expected.
Click Num: