PREreview of StomataCounter: a neural network for automatic stomata identification and counting

UIUC Plant Physiology Journal Club: 2018-09-05

 Iulia Floristeanu,  Cindy Chan , Steven Burgess (0000-0003-2353-7794), Charles Pignon, Stephanie Cullum, Isla Causon, Pietro Hughes AbstractIn the preprint "StomataCounter: a deep learning method applied to automatic stomatal identification and counting” (doi: https://doi.org/10.1101/327494) Fetter et al. introduced a reliable and automated stomata counting program that is more efficient and accurate than human counting and existing algorithms, with a low false positive rate. The authors report the algorithm can be used for previous uncharacterised species and has a 94.2% transfer accuracy when used on untrained datasets. In addition they provide a publically available webtool which could be very beneficial for researchers working on stomata.ReviewWe really enjoyed the paper and found it to be of high interest as the method presented could make the work of a lot of people easier. We were particularly impressed with the precision score of StomataCounter which compares well with other existing algorithms, especially in terms of the transfer accuracy. To our knowledge this is a novel approach, as there are no automated stomata counting technologies available, and it outperforms existing methods. The DCCN appears to overcomes challenges of stomata counting and it correctly identified stomata showing minimal false positives on non-plant and non-stomata covered tissue.The article was well written and easy to follow. We liked the choice of a Deep Convolutional Neural Network (DCNN) for machine learning and felt the authors could have highlighted the benefits of this method by providing a more detailed justification of why it is superior to other algorithms - as this is what made the paper so interesting to us. The text might be further improved by being more specific about results in the abstract and a clearer statement on supplementary data and sampling used.We were interested to know how the algorithm performs on grass species, as they have “dumbbell-shaped” guard cells and companion cells. It was unclear to us whether grass stomata have been used among the training images so we wondered if the accuracy would be the same for this particular shape of stomata. Including some text in the discussion about this would be illuminating. In addition for the sake of reproducibility and to aid readers comprehension it would be useful to provide (1) a complete list of samples analysed and (2) ideally the whole training dataset in a public repository such as Zendo or Dryad, as it could greatly benefit future comparative studiesQuestions we had it would help to clarify

• The authors state “many researchers are likely to manually count stomata” -  is this because other methods are not good enough, user interface is too complicated or just people liking more traditional things?
• In the methods it is written “final fully connected layers by convolutions”. Does this mean there's no fully-connected layers, only convolutional layers? It sounds like both are included in other parts of the paper.
• We were confused by the statement “we argue that the current architecture does not transfer well to different scales and images should be pre-processed to match the training scale of the network,” is that not the reason a fully-convolutional network is used? to avoid the inability to manage different input sizes.

Minor comments

• Consider making legends of Figure 3 and Figure 4 bigger to improve readability

• Figure 5 is very informative and summarizes the data well
• Figure 6 might be improved by including an inset to focus on the down number.
• Figure 7 - scale bars
• There was a strong linear correlation at higher magnification between DCNN and human counts is this a standard magnification for this type of experiment?
• It would be good to include:
• Quantification of the accuracy of the automatic separation between abaxial and adaxial datasets. A failure of separation could potentially have contributed to finding stomata on the "adaxial" cuticle.
• Discussion of whether the magnifications (200x and 400x) are suitable for this kind of analysis.
• Information about why the 4 sources of images were chosen and discussion of whether they provide enough of a range.
• An explanation of why data was which showed <98% was discarded
• Stats on Gingko and poplar datasets.
• Might consider rephrasing “the mean number of stomata detected in the adaxial, aorta, and breast cancer image sets was 1.5, 1.4, and 2.4, respectively, while the  mean value of the abaxial set was 24.1” to better explain the low frequency: to compare with non-stomata dataset vs dataset that has very low number of stomata
• Code availability: It would be helpful if the code and custom scripts (such as separation of abaxial and adaxial leaf sides) are made available and linked to a repository.