文献类型：期刊文献

英文题名：Training-Free Instance Segmentation from Semantic Image Segmentation Masks

作者：Shen, Yuchen[1] Zhang, Dong[2] Zheng, Yuhui[3] Li, Zechao[4] Fu, Liyong[5,6,7] Ye, Qiaolin[1]

第一作者：Shen, Yuchen

机构：[1] The College of Information Science and Technology, Nanjing Forestry University, Jiangsu, Nanjing, 210037, China; [2] The Department of CSE, Hong Kong University of Science and Technology, Hong Kong; [3] The Jiangsu Engineering Center of Network Monitoring, School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, 210044, China; [4] The College of Computer Science and Technology, Nanjing University of Science and Technology, Nanjing, 210094, China; [5] The Research Center of Forestry Remote Sensing and Information Engineering, Central South University of Forestry and Technology, Changsha, 410004, China; [6] The College of Information Science and Technology, Nanjing Forestry University, Nanjing, 210037, China; [7] The Institute of Forest Resource Information Techniques, Chinese Academy of Forestry, Beijing, 100091, China

年份：2023

外文期刊名：arXiv

收录：EI(收录号：20230287823)

语种：英文

外文关键词：Machine learning - Pixels - Semantics

摘要：In recent years, the development of instance segmentation has garnered significant attention in a wide range of applications. However, the training of a fully-supervised instance segmentation model requires costly both instance-level and pixel-level annotations. In contrast, weakly-supervised instance segmentation methods (i.e., with image-level class labels or point labels) struggle to satisfy the accuracy and recall requirements of practical scenarios. In this paper, we propose a novel paradigm for instance segmentation called training-free instance segmentation (TFISeg), which achieves instance segmentation results from image masks predicted using off-the-shelf semantic segmentation models. TFISeg does not require training a semantic or/and instance segmentation model and avoids the need for instance-level image annotations. Therefore, it is highly efficient. Specifically, we first obtain a semantic segmentation mask of the input image via a trained semantic segmentation model. Then, we calculate a displacement field vector for each pixel based on the segmentation mask, which can indicate representations belonging to the same class but different instances, i.e., obtaining the instance-level object information. Finally, instance segmentation results are obtained after being refined by a learnable category-agnostic object boundary branch. Extensive experimental results on two challenging datasets and representative semantic segmentation baselines (including CNNs and Transformers) demonstrate that TFISeg can achieve competitive results compared to the state-of-the-art fully-supervised instance segmentation methods without the need for additional human resources or increased computational costs. The code is available at: TFISeg. Copyright ? 2023, The Authors. All rights reserved.