中文

Journal of Applied Sciences >

2024 , Vol. 42 >Issue 3: 499 - 512

DOI: https://doi.org/10.3969/j.issn.0255-8297.2024.03.011

Computer Science and Applications

A Multi-objective Flow QoS Scheduling Strategy with Improved Proximal Optimization

Expand
  • Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai 200237, China

Received date: 2023-03-07

  Online published: 2024-06-06

Fold

Abstract

Software-defined networking (SDN) can be equipped with flexible flow scheduling strategies to improve the quality of network service systems. However, as the complexity of business traffic increases, existing flow scheduling algorithms may suffer from performance degradation due to decreased scene matching. To address this problem, this paper proposes an intelligent routing strategy based on deep reinforcement learning. The strategy collects various link information through SDN, and implements feature extraction and state awareness based on long-short term memory networks and proximal policy optimization algorithms. The strategy generates dynamic flow scheduling strategies that meet quality of service (QoS) goals in business scenarios, thereby maximizing QoS. Experimental results show that the proposed scheme enhances the QoS index of the entire system by 7.06% compared to existing routing strategies, effectively improving the throughput of the business system.

Key words： deep reinforcement learning; quality of service; routing optimization; software defined network; online scheduling

Cite this article

LIU Xingtong, ZHENG Hong, HUANG Jianhua . A Multi-objective Flow QoS Scheduling Strategy with Improved Proximal Optimization[J]. Journal of Applied Sciences, 2024 , 42(3) : 499 -512 . DOI: 10.3969/j.issn.0255-8297.2024.03.011

References

[1] Agarwal S, Kodialam M, Lakshman T V. Traffic engineering in software defined networks [C]//2013 Proceedings IEEE INFOCOM, 2013: 2211-2219.
[2] Zhang Y, Wei Q S, Chen C, et al. Dynamic scheduling with service curve for QoS guarantee of large-scale cloud storage [J]. IEEE Transactions on Computers, 2018, 67(4): 457-468.
[3] Hartman T, Hassidim A, Kaplan H, et al. How to split a flow? [C]//2012 Proceedings IEEE INFOCOM, 2012: 828-836.
[4] 杨洋, 吕光宏, 赵会, 等. 深度学习在软件定义网络研究中的应用综述[J]. 软件学报, 2020, 31(7): 2184-2204. Yang Y, Lyu G H, Zhao H, et al. Survey on deep learning applications in software defined networking research [J]. Journal of Software, 2020, 31(7): 2184-2204. (in Chinese)
[5] Cui W Z, Qian C. Scalable and load-balanced data center multicast [C]//2015 IEEE Global Telecommunications Conference (GLOBECOM), 2015: 1-6.
[6] Chiesa M, Kindler G, Schapira M. Traffic engineering with equal-cost-multipath: an algorithmic perspective [J]. IEEE-ACM Transactions on Networking, 2017, 25(2): 779-792.
[7] Guo Z Y, Duan J, Yang Y Y. On-line multicast scheduling with bounded congestion in fattree data center networks [J]. IEEE Journal on Selected Areas in Communications, 2014, 32(1): 102-115.
[8] Lee M W, Li Y S, Huang X, et al. Robust multipath multicast routing algorithms for videos in software-defined networks [C]// IEEE International Symposium of Quality of Service (IWQoS), 2014: 218-227.
[9] 柯文龙, 王勇, 叶苗, 等. Ceph云存储网络中一种业务优先级区分的多播流调度方法[J]. 通信学报, 2020, 41(11): 40-51. Ke W L, Wang Y, Ye M, et al. Priority differentiated multicast flow scheduling method in Ceph cloud storage network [J]. Journal on Communications, 2020, 41(11): 40-51. (in Chinese)
[10] 孙茂鑫, 钱红燕, 陈兵, 等. SDN环境下基于MPTCP协议的切换管理[J]. 应用科学学报, 2017, 35(1): 117-127. Sun M X, Qian H Y, Chen B, et al. Handover management based on MPTCP in SDN environment [J]. Journal of Applied Sciences, 2017, 35(1): 117-127.(in Chinese)
[11] Li Y, Gao T, Yang J, et al. Phasic self-imitative reduction for sparse-reward goal-conditioned reinforcement learning [C]//International Conference on Machine Learning (ICML), 2022: 12765-12781.
[12] Su K, Lu Z. Divergence-regularized multi-agent actor-critic [C]//2022 International Conference on Machine Learning (ICML), 2022: 20580-20603.
[13] Lin S C, Akyildiz I F, Wang P, et al. QoS-aware adaptive routing in multi-layer hierarchical software defined networks: a reinforcement learning approach [C]//IEEE International Conference on Services Computing (SCC), 2016: 25-33.
[14] Van Hasselt H, Guez A, Silver D. Deep reinforcement learning with double Q-learning [J]. AAAI Conference on Artificial Intelligence, 2016, 30(1): 2094-2100.
[15] Yu C H, Lan J L, Guo Z H, et al. DROM: optimizing the routing in software-defined networks with deep reinforcement learning [J]. IEEE Access, 2018, 6: 64533-64539.
[16] Tan K, Bremner D, Le Kernec J, et al. Intelligent handover algorithm for vehicle-to-network communications with double-deep Q-learning [J]. IEEE Transactions on Vehicular Technology, 2022, 71(7): 7848-7862.
[17] 孙鹏浩, 兰巨龙, 申涓, 等. 基于牵引控制的深度强化学习路由策略生成[J]. 计算机研究与发展, 2021, 58(7): 1563-1572. Sun P H, Lan J L, Shen J, et al. Pinning control-based routing policy generation using deep reinforcement learning [J]. Journal of Computer Research and Development, 2021, 58(7): 1563-1572. (in Chinese)
[18] Saha N, Misra S, Bera S. Q-flag: QoS-aware flow-rule aggregation in software-defined IoT networks [J]. IEEE Internet of Things Journal, 2022, 9(7): 4899-4906.
[19] Kim G, Kim Y, Lim H. Deep reinforcement learning-based routing on software-defined networks [J]. IEEE Access, 2022, 10: 18121-18133.
[20] Mckeown N, Anderson T, Balakrishnan H, et al. OpenFlow: enabling innovation in campus networks [J]. ACM SIGCOMM Computer Communication Review, 2008, 38(2): 69- 74.
[21] Zhou W, Li L, Luo M, et al. REST API design patterns for SDN northbound API [C]//The 28th International Conference on Advanced Information Networking and Applications Workshops, 2014: 358-365.
[22] Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms [DB/OL]. 2017[2023-03-07]. https://arxiv.org/abs/1707.06347.
[23] Xu Z Y, Tang J, Yin C X, et al. Experience-driven congestion control: when multi-path TCP meets deep reinforcement learning [J]. IEEE Journal on Selected Areas in Communications, 2019, 37(6): 1325-1336.
[24] 王帅, 董育宁, 李涛. 基于LSTM和特征生成的网络流量分类[J]. 应用科学学报, 2022, 40(5): 758-769. Wang S, Dong Y N, Li T. Network traffic classification based on LSTM and feature generation [J]. Journal of Applied Sciences, 2022, 40(5): 758-769. (in Chinese)
[25] Brandt S, Weil S. Ceph: reliable, scalable, and high-performance distributed storage [J]. Annals of Physics (Santa Cruz), 2007: 129-239.
[26] Macedo R, Paulo J, Pereira J, et al. A survey and classification of software-defined storage systems [J]. ACM Computing Surveys (CSUR), 2020, 53(3): 1-38.
[27] Knight S, Nguyen H X, Falkner N R, et al. The Internet topology zoo [J]. IEEE Journal on Selected Areas in Communications, 2011, 29(9): 1765-1775.
Options
Outlines

/

〈 〉