In view of difficulties of edge point detection and edge line linking in edge-based segmentation,this paper proposes a new segmentation method for remote sensing images based on heuristic edge growing.First, improvements are made to the Canny operator for edge point detection including adaptive wavelet de-noising, optimal double-threshold calculation and edge point decision based on total variation. A new heterogeneity index of edge linking is defined, which includes spatial heterogeneity and spectral heterogeneity.Heuristic decision of global mutual best fitting is then proposed to link broken edges of the same target as far as possible. Experiments are carried out using Quickbird image and aerial image to evaluate the proposed method. Segmentation results are compared to those obtained with eCognition, showing that the proposed method can link most edges correctly and produce accurate segmentation results.