PREreview of Characterization of vaginal microbiomes in clinician-collected bacterial vaginosis diagnosed samples

Brief summary of the study

The goal of the study is to compare diagnosis of bacterial vaginosis (BV) between a  PCR-based Labcorp NuSwab® test and 16S v3-v4 rRNA sequencing, evaluating potential for 16S rRNA sequencing to be used as a diagnostic tool. It characterizes the microbial species from patients’ samples initially collected for BV testing withLabcorp NuSwab® test by using 16S rRNA gene sequencing (positive, intermediate and negative samples). 

The findings suggest that BV diagnosis results from 16S rRNA sequencing are compatible with those from the Labcorp NuSwab® test. Although the Labcorp NuSwab® test targets 3 key microorganisms,16S v3-v4 rRNA gene sequencing reveals more than 500 variants from 80 unique genera. The study further reveals other microorganisms that are associated with BV besides three species that are detected by the Labcorp NuSwab® test, which could help with diagnosis of BV. Results also show that BV samples from patients who tested positive for BV had a higher diversity of vaginal bacteria than those that tested intermediate and negative, and that there are many other kinds of vaginal bacteria that are enriched than L. bacilli. 

Major comments (validity or strength of the methodology, experiments and analyses, strength of the conclusions)

• The study lacks detailed background and clinical information about the participants, including how they were selected. Key aspects that are unclear include the study period, study site, and study population composition, such as whether participants were pregnant or non-pregnant. Additionally, it is not specified if the participants were suspected of having bacterial vaginosis (BV) and what criteria were used for this suspicion. It would be helpful to know if this information is published elsewhere.

• The study does not provide any information regarding ethical considerations and consenting from the patients, which is an important aspect of research involving human participants. Without this information, it is unclear how the study addressed these critical ethical issues. 

• The sample should be correctly defined and the age group of women from which the samples were collected in case this preprint is used for further inference studies. It was just mentioned “75 remnant clinician vaginal swabs’’

• Since these are remnant samples, it is important to share information about how they were stored after collection and initial testing with the PCR-based diagnostic kit as well as how long the samples were stored before sequencing.

• It is not clear why two separate sample batches of samples were used for 16S and shotgun characterization. And since background and patient data is missing it is not possible to assess whether the datasets were comparable at all 

• Authors should consider parameters such as sensitivity, specificity, positive and negative predictive values, and the Kappa statistic when comparing two methods for diagnosing bacterial vaginosis (BV)

• The statement, "Metabolic pathway abundances were characterized via metagenomic prediction from amplicon sequence variants (ASVs)," prompts inquiries regarding the limitations of this method in predicting metabolic pathways. These limitations could potentially be addressed using complementary methods such as metabolomics or other appropriate methods. It is essential to recognize any limitations associated with using ASVs for this type of prediction, as understanding these limitations is vital for accurately interpreting the results.

• The findings presented in  Table 1 and Figure 1a were created outside of this paper by someone else by using NuSwab® BV PCR assay thus it should be cited because failing to do so could lead to potential conflicts of interest.

• Were other bacteria, such as Group B Streptococcus (GBS), detected in this study? If so, is there any relationship between their presence and the occurrence of bacterial vaginosis?

• The data is heavy on comparison of key microbial groups/features and not an actual characterization of the vaginal microbiome of the samples as presented in the title and elsewhere in the manuscript. This focus should be captured in the title alongside the fact that the microbial characterization was molecular. This would make the title more informative. 

• Shotgun sequencing results are not discussed. 

Minor comments (clarifications to statements in the text, interpretation of the results, presentation of the data/figures)

• The statement, “NuSwab® assay microbe quantification was strongly concordant with 16 quantification by sequencing (p << 0.01),” includes a double less-than sign (“<<”). It is not clear whether this sign has a specific meaning or if it is an error. Similar errors appear elsewhere in the document. 

• There is a repetition of methods in the introduction and main method section but it will not affect the readability of the paper. 

• The supplementary materials are not easy to follow, especially for results which are supposed to be found there. 

• Authors should add a section with main conclusions of the study

Comments on reporting - information on the statistical analyses or availability of data.

• The methodology used for shotgun data analyses and comparison between 16s and shotgun data are not clear

• The authors do not state whether the data was deposited on Genbank or similar nucleic acid database neither do they provide accession numbers for the same as is customary of molecular data and bioinformatics analyses

Suggestions for future studies

• Address metabolic biomarkers that can assist in diagnosing bacterial vaginosis  using phenotypic methods. 

Inline commenting section

Abstract 

Line “Metabolic pathway abundances were characterized via metagenomic prediction from amplicon sequence variants (ASVs)”.

• Need discussion of PICRUSt2 limitations

• Validation of predicted pathways needed

• Consider discussing alternative approaches

• Address potential bias in pathway predictions

Line “NuSwab assay microbe quantification was strongly concordant with quantification by sequencing (p << 0.01)”

• Unusual statistical notation ("p << 0.01")

• Need explicit concordance measures (e.g., kappa statistics)

• Missing sensitivity/specificity analysis

• No justification for sample size

Introduction 

Line “bacterial vaginosis (BV) poses a significant health burden with a prevalence of 21.2 million women aged 14-49 as of 2004”

• This reference is 20 years old, it would be great to include more recent references, that prevalence level and burden may have changed

Line “Healthy vaginal microbiomes can contain dozens of microbial species in a specific balance”

• It would be great to illustrate (name them) the microbial species in a healthy version. I also wonder when BV happens, can we quantify the healthy microbe species in addition to bacterial growth?

Line “Newer nucleic acid-based clinical tests are now available” 

• It would be great to expand on this in a sentence or two before moving to the proprietary Labcorp technology

Line “many symptomatic women receive negative test results (roughly 50% in some studies)”

• It is a contradictory statement, it would require a significant rationale behind it.

Results 

Line “Sequencing yielded 492 unique 16S V3-V4 Amplicon Sequence Variants (ASVs) that mapped to 83 unique genera”

• Include quality metrics for sequencing data

• Report sequencing depth per sample

• Add error rates and quality control metrics

Line “Of the other 10 bacterial classes identified, six had an average RA of 5% or greater in BV-POS and BV-IND samples”

• Add the species names in text as well (from figure)

Line “Atopobium vaginae formed a small cluster with only Sneathia sanguinegens (Cluster 10), while BVAB-2 clustered with BVAB-3 (Cluster 13) and Megasphaera-1 formed a broad cluster (Cluster 01) with multiple microbes enriched in BV-POS samples (Figure 4c)”

• Do the species in clusters share any similarities? How closely are they related?

Line/subheading “Prediction of a BV-associated metabolic signature”

• Please describe the association between metabolic pathways and vaginal microbes

Line “To better understand the functional capacity of BV microbial signatures, we determined the metagenomic predictions of ASVs and performed differential abundance analysis of the MetaCyc30 metabolic pathways detected using PICRUSt2”

• The authors used PICRUSt2 for metabolic pathway prediction but did not mention whether they utilized the officially recommended visualization package ggpicrust2. ggpicrust2 provides standardized visualization methods specifically designed for PICRUSt2 output, including:

  • Pathway abundance plots

  • Differential abundance analysis visualization

  • Pathway enrichment plots

  • Automated statistical analysis tools

Discussion

Line “75 clinician-collected remnant NuSwab® samples underwent DNA extraction using the ZymoBIOMICS Magbead DNA isolation kit”

• Demographic data of Patients ? Did they have any metabolic disorders? 

• No information about how these samples were selected

Line “A separate set of 54 remnant vaginal swabs were processed through the aforementioned sequencing workflow and through a shotgun metagenomic assay”

• Why were different sample sets used for 16S and shotgun analysis? Why did you not use the same sample set for the three analyses to enable a better comparison and instead use a separate sample set of 54?

• What was the rationale for this split approach?

• How were the samples for shotgun analysis stored and for how long?

• What were the patients’ demographics from whom the samples came from?

• How comparable were these two sample sets?

Line “DADA2 processing and taxonomic analysis”

• The same pipeline was used for 16S and shotgun sequences? I dont see the specific pipeline and analyses for MGx data

Line “CSTs were determined using the most abundant taxon detected, following the criteria used by Ravel et al12 (CST-I: L. crispatus, CST-II: L. gasseri, CST-III: L. iners, CST-IV: diverse communities, CST-V: L. jensenii)”

•  Need justification for using only abundance-based CST classification

• Consider including alternative classification approaches

• Explain why other metrics weren't considered

Data Availability 

Line “Count matrices, sample metadata, ASV taxonomy and sequences, PICRUSt2 predicted pathway abundances, ANCOM-BC microbial differential abundance results, and ALDEx2 pathway differential abundance results are provided in the Supplementary Data”

• Raw sequence data should be deposited in public repository

• Accession numbers needed

• Analysis scripts should be shared

• Detailed bioinformatics pipeline documentation required

Figure Legends 

Line “Figure 4. Differential abundance (DA) analysis and modularized co-occurrence network analysis of BV-POS and BV-NEG samples”

• Figure 4c is particularly complex and difficult to interpret

• Color scheme could be improved for accessibility

• Legend needs more detail

• Scale bars missing in some figures

Conflicts of interest of reviewers

• There are no conflicts of interest to declare.

Competing interests

The authors declare that they have no competing interests.