融合Transformer的剩余使用寿命预测模型

doi:10.3969/j.issn.0255-8297.2024.05.011

应用科学学报 ›› 2024, Vol. 42 ›› Issue (5): 847-856.doi: 10.3969/j.issn.0255-8297.2024.05.011

• 计算机科学与应用 • 上一篇

融合Transformer的剩余使用寿命预测模型

郑红, 刘文, 邱俊杰, 余金浩

华东理工大学信息科学与工程学院, 上海 200237

收稿日期:2023-04-14 发布日期:2024-09-29
通信作者: 郑红，副教授，研究方向为机器学习、形式化方法和智能合约。E-mail:zhenghong@ecust.edu.cn E-mail:zhenghong@ecust.edu.cn
基金资助:
国家重点研发计划（No.2021YFC2701800,No.2021YFC2701801)资助

RUL Prediction Model Combined with Transformer

ZHENG Hong, LIU Wen, QIU Junjie, YU Jinhao

School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, China

Received:2023-04-14 Published:2024-09-29

摘要/Abstract

摘要： 剩余使用寿命（remaining useful life,RUL）预测对大型设备的故障预测与健康管理十分重要。然而，一些设备监测数据具有维度高、规模大、强耦合、参数时变等非线性特征，这些特征会导致RUL预测的准确性较低。为此引入Transformer解码器，并通过多头注意力机制综合全局信息，提出了一种基于多尺度双向长短期记忆网络和Transformer的神经网络模型，以提高模型预测精度。选取航空发动机作为研究对象，使用各个模型在NASA的C-MPASS数据集上进行对比实验，结果表明，在剩余使用寿命预测方面，该文提出的融合Transformer模型的多尺度双向长短期记忆网络模型在准确率和均方根误差指标上均优于其他对比模型。

关键词: 剩余使用寿命, 故障预测与健康管理, 双向长短期记忆网络, Transformer

Abstract: Remaining useful life (RUL) prediction is crucial for prognostics and health management of large equipment. However, nonlinear characteristics such as high dimensionality, large scale, strong coupling, and time-varying parameters in monitoring data of some devices can lead to low accuracy in RUL prediction. To solve this problem, this paper introduces a neural network model that combines a transformer decoder with a multiscale bi-directional long and short-term memory network. This model improves prediction accuracy of the model by integrating global information through a multi-head attention mechanism. Using aviation engines as the research focus, comparative experiments were conducted employing various models on NASA’s C-MPASS dataset. The results show that the proposed multi-scale bi-directional long and short-term memory network fused with Transformer model (MSBiLSTM-Transformer) outperforms other benchmark models, demonstrating superior performance in both accuracy and root mean square error metrics.

Key words: remaining useful life (RUL), prognostics and health management, bi-directional long short-term memory, Transformer

中图分类号:

TP391.9

郑红, 刘文, 邱俊杰, 余金浩. 融合Transformer的剩余使用寿命预测模型[J]. 应用科学学报, 2024, 42(5): 847-856.

ZHENG Hong, LIU Wen, QIU Junjie, YU Jinhao. RUL Prediction Model Combined with Transformer[J]. Journal of Applied Sciences, 2024, 42(5): 847-856.

参考文献

[1] Ramasso E. Investigating computational geometry for failure prognostics [J]. International Journal of Prognostics and Health Management, 2014, 5(1): 1-18.
[2] 朱丽, 杨青, 吴涛, 等. 基于CNN和Bi-LSTM的脑电波情感分析[J]. 应用科学学报, 2022, 40(1): 1-12. Zhu L, Yang Q, Wu T, et al. Emotional analysis of brain waves based on CNN and Bi-LSTM [J]. Journal of Applied Sciences, 2022, 40(1): 1-12. (in Chinese)
[3] Tra V, Nguyen T K, Kim C H, et al. Health indicators construction and remaining useful life estimation for concrete structures using deep neural networks [J]. Applied Sciences, 2021, 11(9): 4113.
[4] Hu C H, Pei H, Si X S, et al. A prognostic model based on DBN and diffusion process for degrading bearing [J]. IEEE Transactions on Industrial Electronics, 2020, 67(10): 8767-8777.
[5] Mo H, Lucca F, Malacarne J, et al. Multi-head CNN-LSTM with prediction error analysis for remaining useful life prediction [C]//27th Conference of Open Innovations Association (FRUCT), 2020, 27(1): 164-171.
[6] Yu W N, Kim I Y, Mechefske C. An improved similarity-based prognostic algorithm for RUL estimation using an RNN autoencoder scheme [J]. Reliability Engineering & System Safety, 2020, 199: 106926.
[7] Chui K T, Gupta B B, Vasant P. A Genetic algorithm optimized RNN-LSTM model for remaining useful life prediction of turbofan engine [J]. Electronics, 2021, 10(3): 285.
[8] Akpudo U E, Jang-Wook H. An automated sensor fusion approach for the RUL prediction of electromagnetic pumps [J]. IEEE Access, 2021, 9: 38920-38933.
[9] Zhao C Y, Huang X Z, Li Y X, et al. A double-channel hybrid deep neural network based on CNN and BiLSTM for remaining useful life prediction [J]. Sensors, 2020, 20(24): 7109.
[10] Bin Shah S R, Chadha G S, Schwung A, et al. A sequence-to-sequence approach for remaining useful lifetime estimation using attention-augmented bidirectional LSTM [J]. Intelligent Systems with Applications, 2021(10/11): 200049.
[11] 王太勇, 王廷虎, 王鹏, 等. 基于注意力机制BiLSTM的设备智能故障诊断方法[J]. 天津大学学报, 2020, 53(6): 601-608. Wang T Y, Wang T H, Wang P, et al. An intelligent fault diagnosis method based on attention-based bidirectional LSTM network [J]. Journal of Tianjin University, 2020, 53(6): 601-608. (in Chinese)
[12] Muneer A, Taib S M, Naseer S, et al. Data-driven deep learning-based attention mechanism for remaining useful life prediction: case study application to turbofan engine analysis [J]. Electronics, 2021, 10(20): 2453.
[13] 邱俊杰, 郑红, 程云辉. 基于多尺度LSTM预测模型研究[J]. 系统仿真学报, 2022, 34(7): 1593-1604. Qiu J J, Zheng H, Cheng Y H. Research on prediction of model based on multi-scale LSTM [J]. Journal of System Simulation, 2022, 34(7): 1593-1604. (in Chinese)
[14] Awad M, Khanna R. Support vector regression [M]. Berkeley, CA: Apress, 2015.
[15] Kang Z Q, Catal C, Tekinerdogan B. Remaining useful life (RUL) prediction of equipment in production lines using artificial neural networks [J]. Sensors, 2021, 21(3): 932.
[16] Ren L, Sun Y Q, Wang H, et al. Prediction of bearing remaining useful life with deep convolution neural network [J]. IEEE Access, 2018, 6: 13041-13049.
[17] Zheng S, Ristovski K, Farahat A, et al. Long short-term memory network for remaining useful life estimation [C]//2017 IEEE International Conference on Prognostics and Health Management (ICPHM), 2017: 88-95.
[18] 王旭, 艾红. 基于CAE与LSTM的航空发动机剩余寿命预测[J]. 北京信息科技大学学报（自然科学版）， 2020, 35(4): 57-62. Wang X, Ai H. Prediction of the remaining life of aerospace engine based on CAE and LSTM [J]. Journal of Beijing Information Science & Technology University, 2020, 35(4): 57-62. (in Chinese)

融合Transformer的剩余使用寿命预测模型

RUL Prediction Model Combined with Transformer

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 4

编辑推荐

Metrics

本文评价

[1]	华怡坦, 黄影平, 过文昊. 基于CNN和Transformer点云图像融合的道路检测[J]. 应用科学学报, 2024, 42(4): 695-708.
[2]	阚亚亚, 张孙杰, 熊娟, 祖奕. 结合transformer多尺度实例交互的稀疏集目标检测[J]. 应用科学学报, 2023, 41(5): 777-788.
[3]	赵宇豪, 陈艳平, 黄瑞章, 秦永彬. 基于跨度回归的中文事件触发词抽取[J]. 应用科学学报, 2023, 41(1): 95-106.
[4]	贾宁, 郑纯军. 融合音频、文本、表情动作的多模态情感识别[J]. 应用科学学报, 2023, 41(1): 55-70.