直觉模糊的结构化最小二乘孪生支持向量机

doi:10.3969/j.issn.0255-8297.2024.02.015

摘要/Abstract

摘要： 针对最小二乘孪生支持向量机（least squares twin support vector machine，LSTSVM）对噪声或是异常数据敏感和忽略数据内在结构信息的问题，提出了一种直觉模糊的结构化最小二乘孪生支持向量机（intuition fuzzy and structural least squares twin support vector machine，IF-SLSTSVM）。首先采用孤立森林对输入样本点进行预处理；然后通过直觉模糊数的概念，赋予输入样本点不同的权重以减少噪声或是异常数据对分类超平面产生的影响；最后采用K-Means算法，以协方差的形式获取输入样本点之间的结构信息。IFSLSTSVM在LS-TSVM的基础上，考虑了输入样本点在特征空间中的分布信息及输入样本点之间的关系，提高了模型的鲁棒性。实验采取UCI数据集，在0%、5%、10%以及20%的不同比例噪声环境对IF-SLSTSVM算法的有效性进行验证。结果显示相较于6种对比算法，IF-SLSTSVM算法有更好的鲁棒性。

关键词: 支持向量机, 孤立森林, 结构信息, 直觉模糊, 聚类, 协方差

Abstract: Addressing the sensitivity of the least squares twin support vector machine(LS-SVM) to noise or abnormal data, and its tendency to overlook intrinsic structural information in the data, this paper introduces an intuition fuzzy and structural least squares twin support vector machine(IF-SLSTSVM). Firstly, the input sample points undergo preprocessing using isolated forest. Subsequently, leveraging the concept of intuitionistic fuzzy, varying weights are assigned to the input sample points to mitigate the impact of noise or abnormal data on the classification hyperplane. Finally, the K-Means algorithm is employed to extract structural information, represented in the form of covariance, among the input sample points. Built upon LS-SVM, IF-SLSTSVM takes into account the distribution information of input sample points in the feature space and their interrelationships,thereby enhancing the model's robustness. Experimental validation is performed using the UCI dataset in noise environments with different proportions of 0%, 5%, 10%, and 20%. The results demonstrate that the IF-SLSTSVM algorithm exhibits superior robustness compared to six other evaluated algorithms.

Key words: support vector machine, isolated forest, structural information, intuition fuzzy, clustering, covariance

中图分类号:

TP391

张法滢, 吕莉, 韩龙哲, 刘东晓, 樊棠怀. 直觉模糊的结构化最小二乘孪生支持向量机[J]. 应用科学学报, 2024, 42(2): 350-363.

ZHANG Faying, LYU Li, HAN Longzhe, LIU Dongxiao, FAN Tanghuai. Intuition Fuzzy and Structural Least Squares Twin Support Vector Machine[J]. Journal of Applied Sciences, 2024, 42(2): 350-363.

参考文献

[1] Vapnik V N. An overview of statistical learning theory [J]. IEEE Transactions on Neural Networks, 1999, 10(5):988-999.
[2] 程鹏宇, 赵嘉, 韩龙哲, 等. 双向多尺度LSTM的短时温度预测[J]. 江西师范大学学报(自然科学版), 2022, 46(2):134-139. Cheng P Y, Zhao J, Han L Z, et al. The short-term temperature prediction based on bidirectional multi-scale LSTM [J]. Journal of Jiangxi Normal University (Natural Sciences Edition), 2022, 46(2):134-139. (in Chinese)
[3] Jayadeva, Khemchandni R, Suresh C. Twin support vector machines for pattern classification [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(5):905-910.
[4] Arun K M, Gopal M. Least squares twin support vector machines for pattern classification [J]. Expert Systems with Applications, 2009, 36(4):7535-7543.
[5] Suykens J A K, Vandewalle J. Least squares support vector machine classifiers [J]. Neural Processing Letters, 1999, 9(3):293-300.
[6] Liu L, Chu M, Gong R, et al. Nonparallel support vector machine with large margin distribution for pattern classification [J]. Pattern Recognition, 2020, 106:107374.
[7] 赵嘉, 姚占峰, 吕莉, 等. 基于相互邻近度的密度峰值聚类算法[J]. 控制与决策, 2021, 36(3):543-552. Zhao J, Yao Z F, Lyu L, et al. Density peaks clustering based on mutual neighbor degree [J]. Control and Decision, 2021, 36(3):543-552. (in Chinese)
[8] 赵嘉, 王刚, 吕莉, 等. 面向流形数据的测地距离与余弦互逆近邻密度峰值聚类算法[J]. 电子学报, 2022, 50(11):2730-2737. Zhao J, Wang G, Lyu L, et al. Density peaks clustering algorithm based on geodesic distance and cosine mutual reverse nearest neighbors for manifold datasets [J]. Acta Electronica Sinica, 2022, 50(11):2730-2737. (in Chinese)
[9] Laxmi S, Gupta S K, Kumar S. Intuitionistic fuzzy least square twin support vector machines for pattern classification [J]. Annals of Operations Research, 2022:1-50.
[10] 吕洁, 麦雄发, 谢妙. 基于二维Gabor小波和孪生支持向量机的图像识别算法[J]. 南京理工大学学报, 2022, 46(1):113-118. Lyu J, Mai X F, Xie M. Image recognition algorithm based on two-dimensional Gabor wavelet and twin support vector machine [J]. Journal of Nanjing University of Science and Technology, 2022, 46(1):113-118. (in Chinese)
[11] Priya V S, Ramyachitra D. Modified genetic algorithm (MGA) based feature selection with mean weighted least squares twin support vector machine (MW-LSTSVM) approach for vegetation classification [J]. Cluster Computing, 2019, 22(6):13569-13581.
[12] Pang J X, Pu X K, Li C G. A hybrid algorithm incorporating vector quantization and one-class support vector machine for industrial anomaly detection [J]. IEEE Transactions on Industrial Informatics, 2022, 18(12):8786-8796.
[13] Dhiman H S, Deb D, Muyeen S M, et al. Wind turbine gearbox anomaly detection based on adaptive threshold and twin support vector machines [J]. IEEE Transactions on Energy Conversion, 2021, 36(4):3462-3469.
[14] 储茂祥, 王安娜, 巩荣芬. 一种改进的最小二乘孪生支持向量机分类算法[J]. 电子学报, 2014, 42(5):998-1003. Chu M X, Wang A N, Gong R F. Improvement on least squares twin support vector machine for pattern classification [J]. Acta Electronica Sinica, 2014, 42(5):998-1003. (in Chinese)
[15] Nasiri J A, Moghadam Charkari N, Mozafari K. Energy-based model of least squares twin Support Vector Machines for human action recognition [J]. Signal Processing, 2014, 104:248-257.
[16] Tanveer M, Khan M A, Ho S S. Robust energy-based least squares twin support vector machines [J]. Applied Intelligence, 2016, 45(1):174-186.
[17] 周裕群, 张德生, 张晓. 一种改进的鲁棒模糊孪生支持向量机算法[J]. 计算机工程与应用, 2023, 59(1):140-148. Zhou Y Q, Zhang D S, Zhang X. Improved robust fuzzy twin support vector machine algorithm [J]. Computer Engineering and Applications, 2023, 59(1):140-148.
[18] Mir A, Nasiri J A. KNN-based least squares twin support vector machine for pattern classification [J]. Applied Intelligence, 2018, 48(12):4551-4564.
[19] Xu R, Wang H. Multi-view learning with privileged weighted twin support vector machine [J]. Expert Systems with Applications, 2022, 206:117787.
[20] 史颂辉, 丁世飞. 基于能量的结构化最小二乘孪生支持向量机[J]. 智能系统学报, 2020, 15(5):1013-1019. Shi S H, Ding S F. Energy-based structural least square twin support vector machine [J]. CAAI Transactions on Intelligent Systems, 2020, 15(5):1013-1019. (in Chinese)
[21] Xie X, Sun F, Qian J, et al. Laplacian Lp norm least squares twin support vector machine [J]. Pattern Recognition, 2023, 136:109192.
[22] 顾丽凤. 引入结构信息的模糊支持向量机算法研究[D]. 保定:河北大学, 2018.
[23] Liu F T, Ting K M, Zhou Z H. Isolation forest [C]//20088th IEEE International Conference on Data Mining, 2008:413-422.
[24] Swetapadma A, Yadav A. A hybrid method for fault location estimation in a fixed series compensated lines [J]. Measurement, 2018, 123:8-18.
[25] Zadeh L A. Fuzzy sets [J]. Fuzzy Sets and Systems, 1965, 8(3):338-353.
[26] Atanassov K T. Intuitionistic fuzzy sets [J]. Fuzzy Sets and Systems, 1986, 20(1):87-96.
[27] Ha M H, Wang C, Chen J Q. The support vector machine based on intuitionistic fuzzy number and kernel function [J]. Soft Computing, 2013, 17(4):635-641.
[28] Xu Z S, Yager R R. Some geometric aggregation operators based on intuitionistic fuzzy sets [J]. International Journal of General Systems, 2006, 35(4):417-433.
[29] Homg D H, Choi C H. Multicriteria fuzzy decision-making problems based on vague set theory [J]. Fuzzy Sets and Systems, 2000, 114(1):103-113.
[30] 李凯, 李娜, 卢霄霞. 一种模糊加权的孪生支持向量机算法[J]. 计算机工程与应用, 2013, 49(4):162-165. Li K, Li N, Lu X X. Twin support vector machine algorithm with fuzzy weighting [J]. Computer Engineering and Applications, 2013, 49(4):162-165. (in Chinese)
[31] Sartakhti J S, Afrabandpey H, Ghadiri N. Fuzzy least squares twin support vector machines [J]. Engineering Applications of Artificial Intelligence, 2019, 85:402-409.
[32] 翟璐璐. 基于结构信息的模糊孪生支持向量机算法研究[D]. 保定:河北大学, 2019.
[33] Hazarika B B, Gupta D. Density weighted twin support vector machines for binary class imbalance learning [J]. Neural Processing Letters, 2022, 54(2):1091-1130.