More than 80 countries have reported malaria cases with 610 thousand deaths and are projected to increase. Identifying malaria early and accurately helps save lives. The effective way to diagnose malaria is through microscopic methods that are labor intensive and require experts with special equipment. Deep learning (DL) has shown promising results in medical diagnosis. Here, we explored various DL models: ResNet18, MobileNetV2, EfficientNet-B2, VGG19 and also proposed a model for detecting malaria presence using blood smears taken from the NIH Malaria dataset. Our experiment shows that MobileNetV2 achieved 96.35% accuracy with the smallest model size (8.49 MB) and fastest inference (1.35 ms). The proposed model achieved 97.67% accuracy, 0.9756 AUC with longest inference time (13.17 ms). The larger architecture outputs a larger model size with moderate accuracy. Upon further pruning, the proposed model gained a slight improvement in accuracy and inference time. The GRAD-CAM, SHAP and LIME shade explainable AI (XAI) insights of the model.