Remaining useful life (RUL) prediction is crucial for prognostics and health management of large equipment. However, nonlinear characteristics such as high dimensionality, large scale, strong coupling, and time-varying parameters in monitoring data of some devices can lead to low accuracy in RUL prediction. To solve this problem, this paper introduces a neural network model that combines a transformer decoder with a multiscale bi-directional long and short-term memory network. This model improves prediction accuracy of the model by integrating global information through a multi-head attention mechanism. Using aviation engines as the research focus, comparative experiments were conducted employing various models on NASA’s C-MPASS dataset. The results show that the proposed multi-scale bi-directional long and short-term memory network fused with Transformer model (MSBiLSTM-Transformer) outperforms other benchmark models, demonstrating superior performance in both accuracy and root mean square error metrics.
ZHENG Hong, LIU Wen, QIU Junjie, YU Jinhao
. RUL Prediction Model Combined with Transformer[J]. Journal of Applied Sciences, 2024
, 42(5)
: 847
-856
.
DOI: 10.3969/j.issn.0255-8297.2024.05.011
[1] Ramasso E. Investigating computational geometry for failure prognostics [J]. International Journal of Prognostics and Health Management, 2014, 5(1): 1-18.
[2] 朱丽, 杨青, 吴涛, 等. 基于CNN和Bi-LSTM的脑电波情感分析[J]. 应用科学学报, 2022, 40(1): 1-12. Zhu L, Yang Q, Wu T, et al. Emotional analysis of brain waves based on CNN and Bi-LSTM [J]. Journal of Applied Sciences, 2022, 40(1): 1-12. (in Chinese)
[3] Tra V, Nguyen T K, Kim C H, et al. Health indicators construction and remaining useful life estimation for concrete structures using deep neural networks [J]. Applied Sciences, 2021, 11(9): 4113.
[4] Hu C H, Pei H, Si X S, et al. A prognostic model based on DBN and diffusion process for degrading bearing [J]. IEEE Transactions on Industrial Electronics, 2020, 67(10): 8767-8777.
[5] Mo H, Lucca F, Malacarne J, et al. Multi-head CNN-LSTM with prediction error analysis for remaining useful life prediction [C]//27th Conference of Open Innovations Association (FRUCT), 2020, 27(1): 164-171.
[6] Yu W N, Kim I Y, Mechefske C. An improved similarity-based prognostic algorithm for RUL estimation using an RNN autoencoder scheme [J]. Reliability Engineering & System Safety, 2020, 199: 106926.
[7] Chui K T, Gupta B B, Vasant P. A Genetic algorithm optimized RNN-LSTM model for remaining useful life prediction of turbofan engine [J]. Electronics, 2021, 10(3): 285.
[8] Akpudo U E, Jang-Wook H. An automated sensor fusion approach for the RUL prediction of electromagnetic pumps [J]. IEEE Access, 2021, 9: 38920-38933.
[9] Zhao C Y, Huang X Z, Li Y X, et al. A double-channel hybrid deep neural network based on CNN and BiLSTM for remaining useful life prediction [J]. Sensors, 2020, 20(24): 7109.
[10] Bin Shah S R, Chadha G S, Schwung A, et al. A sequence-to-sequence approach for remaining useful lifetime estimation using attention-augmented bidirectional LSTM [J]. Intelligent Systems with Applications, 2021(10/11): 200049.
[11] 王太勇, 王廷虎, 王鹏, 等. 基于注意力机制BiLSTM的设备智能故障诊断方法[J]. 天津大学学报, 2020, 53(6): 601-608. Wang T Y, Wang T H, Wang P, et al. An intelligent fault diagnosis method based on attention-based bidirectional LSTM network [J]. Journal of Tianjin University, 2020, 53(6): 601-608. (in Chinese)
[12] Muneer A, Taib S M, Naseer S, et al. Data-driven deep learning-based attention mechanism for remaining useful life prediction: case study application to turbofan engine analysis [J]. Electronics, 2021, 10(20): 2453.
[13] 邱俊杰, 郑红, 程云辉. 基于多尺度LSTM预测模型研究[J]. 系统仿真学报, 2022, 34(7): 1593-1604. Qiu J J, Zheng H, Cheng Y H. Research on prediction of model based on multi-scale LSTM [J]. Journal of System Simulation, 2022, 34(7): 1593-1604. (in Chinese)
[14] Awad M, Khanna R. Support vector regression [M]. Berkeley, CA: Apress, 2015.
[15] Kang Z Q, Catal C, Tekinerdogan B. Remaining useful life (RUL) prediction of equipment in production lines using artificial neural networks [J]. Sensors, 2021, 21(3): 932.
[16] Ren L, Sun Y Q, Wang H, et al. Prediction of bearing remaining useful life with deep convolution neural network [J]. IEEE Access, 2018, 6: 13041-13049.
[17] Zheng S, Ristovski K, Farahat A, et al. Long short-term memory network for remaining useful life estimation [C]//2017 IEEE International Conference on Prognostics and Health Management (ICPHM), 2017: 88-95.
[18] 王旭, 艾红. 基于CAE与LSTM的航空发动机剩余寿命预测[J]. 北京信息科技大学学报（自然科学版）， 2020, 35(4): 57-62. Wang X, Ai H. Prediction of the remaining life of aerospace engine based on CAE and LSTM [J]. Journal of Beijing Information Science & Technology University, 2020, 35(4): 57-62. (in Chinese)
