Early detection of melanoma is crucial for improving patient outcomes, as survival rates decline dramatically with disease progression. Despite significant achievements in deep learning methods for skin lesion analysis, several challenges limit their effectiveness in clinical practice. One of the key issues is the lack of knowledge about the melanoma stage distribution in the training data, raising concerns about the ability of these models to detect early-stage melanoma accurately. Additionally, publicly available datasets that include detailed information on melanoma stage and tumor thickness remain scarce, restricting researchers from developing and benchmarking methods specifically tailored for early diagnosis. Another major limitation is the lack of cross-dataset evaluations. Most deep learning models are tested on the same dataset they were trained on, failing to assess their generalization ability when applied to unseen data. This reduces their reliability in real-world clinical settings. We introduce an early-stage melanoma benchmark dataset to address these issues, featuring images labeled according to T-category based on Breslow thickness. We evaluated several state-of-the-art deep learning models on this dataset and observed a significant drop in performance compared to their results on the ISIC Challenge datasets. This finding highlights the models’ limited capability in detecting early-stage melanoma. This work seeks to advance the development and clinical applicability of automated melanoma diagnostic systems by providing a resource for T-category-specific analysis and supporting cross-dataset evaluation.