Structured PREreview of Comparative Analysis of Supervised Learning Models for Detecting Credit Card and Bank Account Fraud

Does the introduction explain the objective of the research presented in the preprint?

Yes

The Introduction section outlines the study's objectives by first establishing that fraud detection is a vital area of financial analytics where machine learning algorithms are employed to identify suspicious activity and prevent losses. The text then explicitly states that this study investigates the behavior of three machine learning algorithms Logistic Regression, Random Forest, and XGBoost when applied to two datasets with significantly different statistical distributions. The Introduction notes that challenges related to data imbalance and specific fraud patterns remain significant despite advancements in predictive modeling methods.

Are the methods well-suited for this research?

Highly appropriate

The methods implemented are well-suited for the research objective which is to investigate the efficacy and behavior of machine learning models across datasets exhibiting significantly different statistical distributions. Specifically, the study compares three supervised learning models, Logistic Regression, Random Forest, and XGBoost, representing varying complexity, on two carefully selected datasets. One dataset is a synthetic, balanced set (50:50 ratio) designed to assess model performance under idealized conditions, while the second is a real-world Bank Account Fraud Dataset featuring extreme class imbalance (approximately 1% fraud). To maintain rigorous comparison, the research ensures that preprocessing methodologies, sampling techniques, and evaluations are kept constant across models, which isolates the effect of dataset structure on model stability and behavior. The use of stratified 60/20/20 data splits and the targeted application of SMOTE exclusively to the training set of the imbalanced data are appropriate techniques for managing class imbalance and preventing data leakage. Lastly, the choice of evaluation metrics, F1-score and AUC-ROC, is highly suitable for accurately assessing model performance in scenarios characterized by high class imbalance.

Are the conclusions supported by the data?

Highly supported

The conclusions of the research are consistently supported by the data presented, which isolates the critical impact of data structure on model performance in financial fraud detection. The data shows that when tested on the synthetic, balanced Credit Card dataset, all three supervised models achieved near perfect outcomes, with XGBoost attaining an F1 score of 99.98% and an AUC-ROC of 99.99%, confirming the conclusion that idealized balanced datasets produce nearly perfect results. However, the sharp performance drop observed when applying the same models to the real world imbalanced Bank Account Fraud Dataset validates the central conclusion that data structure and imbalance restrict performance limits more than algorithm complexity. Furthermore, XGBoost's mean F1 score dropped significantly from 99.97% to 23.41% on the imbalanced data, illustrating the dramatic impact of class imbalance. Nevertheless, the conclusion that the XGBoost model yielded the best balance of high precision and good recall is confirmed by its highest measured F1 score (23.41%) and AUC-ROC (89.34%) on the complex imbalanced dataset. Moreover, the final discussion that moderate F1 performance is consistent with high business value under realistic class imbalance is supported by the context that F1 scores between 15 to 25% align with real world operational benchmarks for detecting fraudulent behavior.

Are the data presentations, including visualizations, well-suited to represent the data?

Highly appropriate and clear

The data presentation is highly suitable for representing the quantitative findings of the research, although the sources primarily rely on structured tables rather than explicit visualizations. The results sections utilize specific tables to clearly convey the performance metrics for Logistic Regression, Random Forest, and XGBoost on both the balanced and imbalanced datasets. These tables effectively present crucial evaluation metrics such as the F1-score, AUC-ROC, and CV Stability for each model, which is essential for the comparative analysis central to the study. Additionally, a Comparative Summary table is employed to precisely highlight the significant performance degradation, specifically quantifying the decrease in Mean F1 and Mean AUC when the XGBoost model shifted from the balanced to the imbalanced dataset, directly supporting the study’s core conclusions about data structure impact.

How clearly do the authors discuss, explain, and interpret their findings and potential next steps for the research?

Very clearly

The author clearly discuss, explain, and interpret their findings, concluding that data structure, specifically class imbalance, is the primary driver restricting performance limits in financial fraud detection rather than total algorithm complexity. They explain that the balanced synthetic dataset produced near perfect separation for all algorithms, contrasting sharply with the significantly lower, yet realistic, F1 scores produced by the highly imbalanced dataset. The interpretation highlights that the "low" F1 scores (15 to 25%) achieved on the imbalanced data should be viewed not as poor statistical performance, but as high grade, real world operational scores consistent with accepted benchmarks, given that fraud transactions invariably compose less than 1% of total records and the objective is satisfactory detectability. Regarding next steps, the research explicitly recommends a two tier benchmark framework for future work, utilizing both idealized and realistic datasets. Future directions are clearly outlined, suggesting the extension of the work by looking at adaptive resampling, dynamic thresholds, cost sensitivity, focal loss training, temporal and sequential modeling (like LSTM and Transformers), and Explainable AI methods (SHAP, LIME) to enhance real world application and bridge the gap between laboratory and live deployment performance.

Is the preprint likely to advance academic knowledge?

Highly likely

The preprint is likely to advance academic knowledge by providing a rigorous comparative analysis that isolates the critical influence of data structure, specifically class imbalance, on the performance limits of supervised learning models in financial fraud detection.

Would it benefit from language editing?

Yes

While the text is generally clear, certain phrasing and sentence structures reveal that the manuscript could benefit from further language editing to enhance clarity and academic precision. Specific constructions such as "realistic of these are rare" and the somewhat awkward passive phrasing like "should not be felt to indicate a poor performance of the statistical model employed" could be tightened. Additionally, sentences discussing the operational context, for example, "The model of operation, where fraud detection was involved, was not to achieve full accuracy," might be simplified for better flow, confirming that additional refinement could optimize the manuscript's linguistic quality.

Would you recommend this preprint to others?

Yes, it’s of high quality

This preprint is highly recommended, particularly for individuals focused on machine learning in financial analytics and imbalanced classification problems, because it provides an in-depth comparative analysis that directly advances academic knowledge.

Is it ready for attention from an editor, publisher or broader audience?

Yes, after minor changes

While the author previously used an AI tool for initial language refinement, the manuscript would benefit from further minor language editing to enhance academic precision and tighten certain phrasing for optimal clarity before publication.

Competing interests

The author declares that they have no competing interests.

Use of Artificial Intelligence (AI)

The author declares that they did not use generative AI to come up with new ideas for their review.