Abstract: Existing series of loss functions based on intersection over union (IoU) have certain limitations, impacting the accuracy and stability of bounding box regression in object detection. To address this problem, a bounding box regression loss based on nonlinear Gaussian squared distance is proposed. Firstly, the three factors including overlapping, center point distance and aspect ratio in the bounding box are comprehensively considered, and the bounding box is modeled as a Gaussian distribution. Then a Gaussian squared distance is proposed to measure the distance between two distributions. Finally, a nonlinear function is designed to transform the Gaussian square distance into a loss function that facilitates neural network learning. Experimental results show that compared with IoU loss, the mean average precision of the proposed method on mask region-based convolutional neural network, fully convolutional one-stage object detector and adaptive training sample selection object detector is improved by 0.3%, 1.1% and 2.3%, respectively. These results demonstrate the efficiency of the proposed method in enhancing target detection performance and supporting the regression of high-precision bounding boxes.

Key words: object detection, bounding box regression, Gaussian distribution, intersection over union (IoU), convolutional neural network