Saltar al contenido principal

Escribe una PREreview

HEADHUNTER: Training-Free Annotated Dataset Synthesis via Self-Guided Diffusion Transformer Attention Head Selection

Publicada
Servidor
Preprints.org
DOI
10.20944/preprints202606.2132.v1

Pixel-level annotation remains a major bottleneck for semantic segmentation, motivating methods that synthesize image-label pairs directly from generative models. Prior synthetic dataset generators typically obtain pseudo-labels from cross-attention maps or learned decoders over generative features; however, recent text-to-image (T2I) models increasingly use multimodal diffusion transformers (MM-DiTs), where concept localization is no longer exposed through a single cross-attention pathway but instead distributed across many layers and attention heads. Existing MM-DiT localization methods address this by aggregating saliency across heads, but we observe that this averaging can dilute clean target localizers due to attention head heterogeneity. We introduce HEADHUNTER, a training-free, open-vocabulary segmentation framework that uses aggregate concept saliency as a self-guided proxy to select the single attention head that best localizes a queried textual concept, yielding cleaner segmentation masks. We then use HEADHUNTER to turn target classes into training data automatically: a large language model (LLM) diversifies prompts, an MM-DiT generates images, HEADHUNTER produces pseudo-labels, and a vision language model (VLM) verifies each image-mask pair before acceptance. On PASCAL VOC2012, HEADHUNTER achieves 79.2 mIoU in zero-shot single-class segmentation, outperforming MM-DiT head aggregation. Using only our generated image-label pairs, DeepLabV3 with a ResNet-101 backbone reaches 64.9 mIoU on VOC2012 validation, matching or outperforming comparable synthetic dataset generators and showing that the proposed pipeline produces effective dense labels without human intervention. Code: https://github.com/Rohanl2298/HEADHUNTER.

Puedes escribir una PREreview de HEADHUNTER: Training-Free Annotated Dataset Synthesis via Self-Guided Diffusion Transformer Attention Head Selection. Una PREreview es una revisión de un preprint y puede variar desde unas pocas oraciones hasta un extenso informe, similar a un informe de revisión por pares organizado por una revista.

Antes de comenzar

Te pediremos que inicies sesión con tu ORCID iD. Si no tienes un iD, puedes crear uno.

¿Qué es un ORCID iD?

Un ORCID iD es un identificador único que te distingue de otros/as con tu mismo nombre o uno similar.

Comenzar ahora