Skip to main content

Write a PREreview

3SGAN: Semisupervised and Multitask GAN for Stain Normalization and Nuclei Segmentation of Histopathological Images

Posted
Server
Preprints.org
DOI
10.20944/preprints202512.0031.v1

Background/Objectives: Variations in staining styles—arising from differences in tissue preparation, scanners, and laboratory protocols—severely compromise the robustness of automated cell segmentation algorithms in digital pathology. Moreover, manual nucleus annotation is extremely labor-intensive, leading to a scarcity of large-scale, fully anno-tated datasets for supervised nucleus segmentation. This study proposes a novel framework that simultaneously mitigates staining variability and achieves high-accuracy nucleus segmentation using only minimal annotations. Methods: We present 3SGAN, a multitask dual-branch generative adversarial network (GAN) that jointly performs stain normalization and nucleus segmentation in a semi-supervised manner. The framework adopts a teacher-student paradigm: a lightweight teacher model (AttCycle) equipped with attention gates generates reliable pseudo-labels, while a high-capacity student model (TransCycle) leveraging a hybrid CNN-Transformer architecture further refines performance. 3SGAN was trained and evaluated on a large dataset of 1,408 Whole-Slide Images (WSIs) from two medical institutions, encompassing 101 distinct staining styles, with nucleus-level annotations required for only 5% of the data. Results: 3SGAN sig-nificantly outperformed state-of-the-art methods, achieving substantial improvements in stain normalization quality (RMSE, PSNR, SSIM) and marked gains in nucleus seg-mentation performance (F1 score, mean IoU, and AJI). External validation on independent MoNuSeg and PanNuke datasets, as well as on previously untested tumor-rich non-ROI regions from our in-house WSIs, confirmed strong generalizability with excellent stain normalization and top-tier segmentation accuracy across diverse staining protocols, tissue types, and pathological patterns. Conclusions: The proposed 3SGAN framework demonstrates that high-performance nucleus segmentation and stain normalization can be achieved with minimal annotation requirements, offering a practical and scalable solution for digital pathology applications across diverse clinical settings and staining protocols.

You can write a PREreview of 3SGAN: Semisupervised and Multitask GAN for Stain Normalization and Nuclei Segmentation of Histopathological Images. A PREreview is a review of a preprint and can vary from a few sentences to a lengthy report, similar to a journal-organized peer-review report.

Before you start

We will ask you to log in with your ORCID iD. If you don’t have an iD, you can create one.

What is an ORCID iD?

An ORCID iD is a unique identifier that distinguishes you from everyone with the same or similar name.

Start now