论文部分内容阅读
传统栈式稀疏自动编码机的全局权值微调算法基于反向传播,由于深度神经网络损失函数的非凸性质,反向传播算法极易陷入局部极小值,且前几层网络的权值往往得不到充分训练。为此引入极限学习算法到深度神经网络的训练中来,将多个通过二阶Hessian-Free方法训练后的稀疏自动编码机层叠起来,然后通过正则化ELM算法确定最后输出层的权值。结果表明二阶Hessian-Free方法克服了最速下降法存在的锯齿型收敛问题,并保留了牛顿方法计算Hessian矩阵的时间和空间开销。将机车振动数据分别用离散傅里叶变换和量子傅里叶变换转换到频域后作为特征输入网络,检测结果表明网络的泛化能力和训练速度都有了可观的提升。
Due to the non-convex nature of the loss function of the deep neural network, the backpropagation algorithm can easily fall into the local minima, and the weight of the first few layers of networks tends to be Not fully trained. For this reason, we introduce the limit learning algorithm to the training of deep neural network, and then we concatenate several sparse auto-coders trained by the second-order Hessian-Free method and then determine the weight of the final output layer by the regularized ELM algorithm. The results show that the second-order Hessian-Free method overcomes the jagged convergence problem of the steepest descent method and preserves the time and space cost of the Newton method for calculating the Hessian matrix. The locomotive vibration data is transformed into the frequency domain by using discrete Fourier transform and quantum Fourier transform, respectively. The results show that the network generalization ability and training speed have been improved significantly.