多任务模糊聚类驱动的多任务TSK模糊系统模型

doi:10.3969/j.issn.0255-8297.2020.05.007

摘要/Abstract

摘要： 该文提出了一种多任务Takagi-Sugeno-Kang（TSK）模糊系统建模方法.首先给出了一种新的多任务模糊c均值聚类算法，能够有效提取所有任务之间的公共信息和每个任务的私有信息，进而利用所得的聚类中心构建多任务TSK模糊系统的前件参数.其次设计了一种具备多任务协同学习机制的后件参数优化方法，可以优化多任务TSK模糊系统的后件参数.最后基于优化的前后件参数，构建出具体多任务模糊聚类方法驱动的多任务TSK模糊系统模型（multi-task fuzzy c-means based multi-task TSK fuzzy system，MTFCM-MT-TSK-FS）以用于实际应用.分别在合成和真实数据集上进行实验，结果验证了该模型的有效性.

关键词: 多任务学习, 多任务模糊c均值, 多任务协同学习, 多任务Takagi-Sugeno-Kang模糊系统式

Abstract: In this paper, a novel modeling approach of multi-task Takagi-Sugeno-Kang (TSK) fuzzy system is presented. Firstly, we propose a new multi-task fuzzy c-means clustering algorithm, which is used to extract the public information between all tasks and the private information of each task effectively. Accordingly, the antecedent parameters of multi task TSK fuzzy system can be constructed with the obtained clustering centers. Secondly, a novel consequent parameters optimization method of the multi-task TSK fuzzy system is proposed based on the multi-task collaborative learning mechanism. Finally, a practical application oriented multi-task TSK fuzzy system is completed based on the two proposed algorithms. Experimental results on several synthetic and real-world datasets demonstrate the validity of the proposed model.

Key words: multi-task learning, multi-task fuzzy c-means, multi-task collaborative learning, multi-task Takagi-Sugeno-Kang fuzzy systems

中图分类号:

P751.1

蒋亦樟, 华蕾, 张群, 钱鹏江, 夏开建. 多任务模糊聚类驱动的多任务TSK模糊系统模型[J]. 应用科学学报, 2020, 38(5): 742-760.

JIANG Yizhang, HUA Lei, ZHANG Qun, QIAN Pengjiang, XIA Kaijian. Multi-task Fuzzy Clustering Based Multi-task TSK Fuzzy System[J]. Journal of Applied Sciences, 2020, 38(5): 742-760.

参考文献

[1] 黄一鸣, 雷航, 李晓瑜. 量子机器学习算法综述[J]. 计算机学报, 2018, 41(1):145-163. Huang Y M, Lei H, Li X Y. Review of quantum machine learning algorithms[J]. Chinese Journal of Computers, 2018, 41(1):145-163. (in Chinese)
[2] 汤深伟, 贾瑞玉. 基于改进粒子群算法的k均值聚类算法[J]. 计算机工程与应用, 2019, 55(18):140-145. Tang S W, Jia R Y. K-means clustering algorithm based on improved particle swarm optimization algorithm[J]. Computer Engineering and Applications, 2019, 55(18):140-145. (in Chinese)
[3] 刘洋, 王慧琴, 张小红. 结合蚁群算法的改进粗糙K均值聚类算法[J]. 数据采集与处理, 2019, 34(2):341-348. Liu Yang, Wang H Q, Zhang X H. Improved rough K-means clustering algorithm combined with ant colony algorithm[J]. Journal of Data Acquisition & Processing, 2019, 34(2):341-348. (in Chinese)
[4] Bezdek J C. Pattern recognition with fuzzy objective function algorithms[J]. New York:Plenum Press, 1981.
[5] Jiang Y, Chung F L, Wang S. Enhanced fuzzy partitions vs data randomness in FCM[J]. Journal of Intelligent and Fuzzy Systems, 2014, 27(4):1639-1648.
[6] 高云龙, 杨程宇, 王志豪, 等. 簇间可分的鲁棒模糊C均值聚类算法[J]. 电子与信息学报, 2019, 5(1):1114-1121. Gao Y L, Yang C Y, Wang Z H, et al. Robust fuzzy C-means clustering algorithm for separable clusters[J]. Journal of Electronics and Information, 2019, 5(1):1114-1121. (in Chinese)
[7] Karayiannis N B, Mi G W. Growing radial basis neural networks:Merging supervised and unsupervised learning with network growth techniques[J]. IEEE Transactions on Neural Networks, 1997, 8(6):1492-1506.
[8] 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6):1229-1251. Zhou F Y, Jin L P, Dong J. A review of convolution neural networks[J]. Chinese Journal of Computers, 2017, 40(6):1229-1251. (in Chinese)
[9] 安亚利, 周水生, 陈丽, 等. 鲁棒支持向量机及其稀疏算法[J]. 西安电子科技大学学报, 2019, 46(1):64-72. An Y L, Zhou S S, Chen L, et al. Robust support vector machine and its sparse algorithm[J]. Journal of Xi'an University of Electronic Science and Technology, 2019, 46(1):64-72. (in Chinese)
[10] Keerthi S S, Shevade S K, Bhattacharyya C, et al. Improvements to Platt's SMO algorithm for SVM classifier design[J]. Neural Computation, 2001, 13(3):637.
[11] 谢文博, 李鹤, 张孟禹, 等. 前件变量未知的T-S模糊系统输出反馈控制[J]. 控制与决策, 2019, 34(9):1876-1884. Xie W B, Li H, Zhang M Y, et al. Output feedback control of Tmurs fuzzy system with unknown antecedent variables[J]. Control and Decision, 2019,34(9):1876-1884. (in Chinese)
[12] 顾晓清, 王士同, 倪彤光, 等. 具有强解释性的贝叶斯MA型模糊系统[J]. 控制与决策, 2018, 33(1):16-26. Gu X Q, Wang S T, Ni T G, et al. Bayesian MA fuzzy system with strong interpretation[J]. Control and Decision, 2018, 33(1):16-26. (in Chinese)
[13] 顾晓清, 王士同. 知识嵌入的贝叶斯MA型模糊系统[J]. 计算机研究与发展, 2017, 54(5):998-1011. Gu X Q, Wang S T. Knowledge embedded Bayesian MA fuzzy system[J]. Computer Research and Development, 2017, 54(5):998-1011. (in Chinese)
[14] Deng Z H, Jiang Y Z, Choi K S, et al. Knowledge-leverage-based TSK fuzzy system modeling[J]. IEEE Transactions on Neural Networks and Learning Systems, 2013, 24(8):1200-1212.
[15] 杭文龙, 梁爽, 刘解放, 等. 贝叶斯L2型TSK模糊系统[J]. 控制与决策, 2017, 32(10):1871-1878. Hang W L, Liang S, Liu J F, et al. Bayesian L 2 TSK fuzzy system[J]. Control and Decision, 2017, 32(10):1871-1878. (in Chinese)
[16] Jiang Y, Deng Z, Chung F L, et al. Multi-task TSK fuzzy system modeling using inter-task correlation information[J]. Information Sciences, 2015, 298:512-533.
[17] Jiang Y, Chung F L, Ishibuchi H. Multitask TSK fuzzy system modeling by mining intertask common hidden structure[J]. IEEE Transactions on Cybernetics, 2015, 45(3):548-561.
[18] Kinoshita S, Udaka S, Shimamoto M. Studies on amino acid fermentation, Part I. production of L-glutamic acid by various microorganisms[J]. Journal of General and Applied Microbiology, 1957, 3:193-205.
[19] Caruana R. Multitask learning[J]. Machine Learning, 1997, 28(1):41-75.
[20] 史荧中, 邓赵红, 钱鹏江, 等. 基于共享矢量链的多任务概念漂移分类方法[J]. 控制与决策, 2018, 33(7):1215-1222. Shi Y Z, Deng Z H, Qian P J, et al. Classification method of multitask concept drift based on shared vector chain[J]. Control and Decision, 2018, 33(7):1215-1222. (in Chinese)
[21] Ji Y, Sun S. Multitask multiclass support vector machines:model and experiments[J]. Pattern Recognition, 2013, 46(3):914-924.
[22] Gu Q, Zhou J. Learning the shared subspace for multi-task clustering and transductive transfer classification[C]//Proceedings of the International Conference Data Mining (ICDM). Miami, USA, 2009:59-168.
[23] 张晓彤, 张宪超, 刘晗. 基于特征和实例迁移的加权多任务聚类[J]. 计算机学报, 2019, 42(12):2614-2630. Zhang X T, Zhang X C, Liu H. Weighted multitask clustering based on feature and instance migration[J]. Chinese Journal of Computers, 2019, 42(12):2614-2630. (in Chinese)
[24] Zhang Z, Zhou J. Multi-task clustering via domain adaptation[J]. Pattern Recognition, 2012, 45(1):465-473.
[25] Fang J, Ji S, Xue Y, et al. Multitask classification by learning the task relevance[J]. IEEE Signal Processing Letters, 2008, 15:593-596.
[26] 张苗辉, 张博, 高诚诚. 一种多任务的卷积神经网络目标分类算法[J]. 激光与光电子学进展, 2019, 23:222-229. Zhang M H, Zhang B, Gao C C. A multitasking convolution neural network target classification algorithm[J]. Advances in Laser and Optoelectronics, 2019, 23:222-229. (in Chinese)
[27] Wang H. A new sparse multi-task regression and feature selection method to identify brain imaging predictors for memory performance[C]//Proceedings of the IEEE International Conference on Computer Vision, 2011:557-572.
[28] Solnon M, Arlot S, Bach F. Multi-task regression using minimal penalties[J]. Journal of Machine Learning Research, 2012, 13:2773-2812.
[29] Zhou J, Yuan L, Liu J, et al. A multi-task learning formulation for predicting disease progression[C]//Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, USA, 2011:21-24.
[30] 任建华, 赵凯龙, 刘欣宜. 模糊控制理论在风电机组变桨控制系统中的应用[J]. 现代电子技术, 2019, 42(23):130-134. Ren J H, Zhao K L, Liu X Y. Application of fuzzy control theory in variable pitch control system of wind turbine[J]. Modern Electronic Technology, 2019, 42(23):130-134. (in Chinese)
[31] Chung F L, Deng Z H, Wang S T. An adaptive fuzzy-inference-rule-based flexible model for automatic elastic image registration[J]. IEEE Transactions on Fuzzy Systems, 2009, 17(5):995-1010.
[32] 张代远, 吕鹏. 基于零代价函数神经网络的神经-模糊系统与应用[J]. 计算机应用, 2006, 26(A2):208-210. Zhang D Y, Lü P. Neural-fuzzy system based on zero-cost function neural network and its application[J]. Computer applications, 2006, 26(A2):208-210. (in Chinese)
[33] Wang N, Meng J E. Direct adaptive fuzzy tracking control of marine vehicles with fully unknown parametric dynamics and uncertainties[J]. IEEE Transactions on Control Systems Technology, 2016, 24(5):1845-1852.
[34] 王华强, 张博扬. 基于AFSA-BP算法的模糊PID控制器在DMF回收系统中的应用[J]. 测控技术, 2019, 38(10):80-84. Wang H Q, Zhang B Y. Application of fuzzy PID controller based on AFSA-BP algorithm in DMF recovery system[J]. Measurement and Control Technology, 2019, 38(10):80-84. (in Chinese)
[35] Deng Z H, Choi K S, Chung F L, et al. Scalable TSK fuzzy modeling for very large datasets using minimal-enclosing-ball approximation[J]. IEEE Transactions on Fuzzy Systems, 2011, 19(2):210-226.
[36] Juang C F, Chiu S H, Shiu S J. Fuzzy system learned through fuzzy clustering and support vector machine for human skin color segmentation[J]. IEEE Transactions on Systems, Man, and Cybernetics-Part A:Systems and Humans, 2007, 37(6):1077-1087.
[37] Juang C F, Hsieh C D. TS-fuzzy system-based support vector regression[J]. Fuzzy Sets and Systems, 2009, 160(17):2486-2504.
[38] Krysmann M. Takagi-Sugeno-Kanga fuzzy fusion in dynamic multi-classifier system[C]//Proceedings of the 2nd World Congress on Electrical Engineering and Computer Systems and Science (EECSS'16), 2016:108.1-108.7.
[39] Ayres A, Zuben V. Multitask learning applied to evolving fuzzy-rule-based predictors[J]. Evolving Systems, 2019. doi:10.1007/s12530-019-09300-w.
[40] Wang J, Lin D, Deng Z, et al. Multitask TSK fuzzy system modeling by jointly reducing rules and consequent parameters[J]. IEEE Transactions on Systems, Man, and Cybernetics:Systems, 2019. doi:10.1109/TSMC.2019.2930616.
[41] Chiu S L. Fuzzy model identification based on cluster estimation[J]. Journal of Intelligent and Fuzzy Systems, 1994, 2(3):267-278.
[42] Leski J M. Fuzzy c-means clustering and its application to a fuzzy rule-based classifier:toward good generalization and good interpretability[J]. IEEE Transactions on Fuzzy Systems, 2015, 23(4):802-812.
[43] Ito K, Nakano R. Optimizing support vector regression hyperparameters based on crossvalidation[C]//Proceedings of the International Joint Conference on Neural Networks, 2003:2077-2082.
[44] Fan R E, Chen P H, Lin C J. Working set selection using second order information for training support vector machines[J]. Journal of Machine Learning Research, 2005, 6:1889-1918.
[45] Kim D, Sra S, Dhillon I S. A scalable trust-region algorithm with application to mixednorm regression[C]//Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010:21-24.