Abstract: A flexible Boolian mask operation merged algorithm called A algorithm has been developed. It is suitable for a large amount of mask data, including diagonal edges that have any angle. Calculation of Boolian mask combination between different layers of an integrated circuit is a basic procedure in designing rule checking, connectivity checking and device recognition from the layout. A new and clear figure definition and a concise datastructure are used in A algorithm, making the algorithm treatments for various Boolian operations merged and unified. Furthermore, it is easy to design various Boolian operation programs by using A algorithm.