PolyPVT-MambaNet: A Multi-scale Attention and Mamba-Enhanced Framework for Polyp Segmentation

Abstract

A polyp is an unusual tissue outgrowth that extends from the surface of an internal structure, most commonly found on the mucosal surfaces of different organs. Although most are harmless, some can gradually transform into malignant tumors, increasing the probability of serious health problems. Polyps segmentation plays an essential role in establishing a foundation for the development of related methods and tasks for the identification of polyps, helping doctors analyze polyp images with greater precision. With recent advances in neural network models, particularly those based on Transformer and CNN architectures, the precision of polyp segmentation has improved significantly. However, the diagnosis of polyps is a challenging task because of the variation in the size and shape of polyps, which requires more innovation to enhance output results. To improve performance with this challenge, we proposed a polyp segmentation model, named PolyPVT-MambaNet, based on CNN, Transformer, and Mamba architectures. In this study, we designed the Tri-path Multi-scale Attention for calculating attention weights by capturing cross-dimension interaction and Multi-scale-Fusion Mamba (MFM) Block combined with Dynamic Convolution in the decoder. Our experiment of the polyps dataset, especially on the Kvasir-SEG and CVC-ClinicDB sets, shows the potential results and outperforms other state-of-the-art models. The code for our method will be available at: https://github.com/nvquang021/MambaPolypSeg.