Abstract: UAV-assisted communication system is an important component of future wireless networks. In order to further improve the utilization of time-frequency resources in UAV-assisted communication systems, this paper proposes a communication architecture based on non-orthogonal multiple access (NOMA) technology and introduces a TD3-TOPATM (twin delayed-trajectory optimization and power allocation for total throughput maximization) algorithm based on the double-delay deep deterministic policy gradient strategy. The TD3-TOPATM algorithm jointly optimizes the 3D trajectory and power allocation strategy of the UAV, with the aim of maximizing the total throughput while satisfying constraints on maximum power, spatial boundaries maximum flight speed, and quality of service (QoS). Simulation results show that compared with the trajectory optimization algorithm with random optimization, the TD3-TOPATM algorithm achieves a performance gain of 98%. Additionally, it outperforms the deep Q-network (DQN)-based trajectory optimization and resource allocation algorithm, increasing total throughput by 19.4%, and surpasses the deep deterministic policy gradient (DDPG)-based algorithm with a 9.7% throughput gain. Furthermore, the NOMA-based UAV-assisted communication scheme achieves a 55% performance gain compared to the OMA-based scheme.

Key words: deep reinforcement learning, UAV-assisted communication, 3D trajectory optimization, non-orthogonal multiple access, double-delay deep deterministic policy gradient