Work

Cluster2Former: Semisupervised Clustering Transformers for Video Instance Segmentation

Year: 2024

Type: article

Abstract: A novel approach for video instance segmentation is presented using semisupervised learning. Our Cluster2Former model leverages scribble-based annotations for training, significantly reducing the need for comprehensive pixel-level masks. We augment a video instance segmenter, for example, the Mask2Former architecture, with similarity-based constraint loss to handle partial annotations efficiently. We demonstrate that despite using lightweight annotations (using only 0.5% of the annotated pixels), Cluster2Former achieves competitive performance on standard benchmarks. The approach offers a cost-effective and computationally efficient solution for video instance segmentation, especially in scenarios with limited annotation resources.

Source: Sensors

DOI: 10.3390/s24030997

Authors: Áron Fóthi, Adrián Szlatincsán, Ellák Somfai

Institutions: Eötvös Loránd University, HUN-REN Wigner Research Centre for Physics

Language: English

Topic: Advanced Neural Network Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,955

Funders: Mesterséges Intelligencia Nemzeti Laboratórium, European Commission, Nemzeti Kutatási, Fejlesztési és Innovaciós Alap, National Research, Development and Innovation Office