Unified Network With Detail Guidance for Panoptic Segmentation

Panoptic segmentation has won popularity in image perception for its unique advantages. A generic backbone is utilized to extract image features, either fusing semantic and instance segmentation results or end-to-end. Backbone is able to merge low-level details and high-level semantics. However, in practice, detailed information is weakened after deep convolutions. To address this limitation, we propose a novel unified network consisting of a bilateral feature extraction structure and an aggregation module. Both detail and semantic information extraction are decoupled successfully. Specifically, the bilateral feature extraction structure comprises two main branches. One branch uses a generic backbone to obtain the rich receptive field, while the other uses the guidance of detail ground-truth to extract low-level features. Furthermore, the aggregation module combines the results of two branches to obtain a large receptive field with detailed information. Comparative experiments are performed on COCO and Cityscapes datasets. The results demonstrate that high accuracy is obtained. Among them, 41.3 panoptic quality is achieved on COCO, and 59.9 is achieved on Cityscapes.