PREreview of COVID-19 risk score as a public health tool to guide targeted testing: A demonstration study in Qatar

In this study, the authors report the development and validation of a prediction model of "risk of COVID-19" in Qatar, with further updating and evaluation of the model. It is not clear what the authors intended to predict, since they only mention "risk of COVID-19", but they later mention that the model was meant to predict "risk of exposure to SARS-CoV-2", which doesn't necessarily imply that the individual will have a positive test since having a positive test is dependent on the course of disease and time of testing is very important for this. Adding confusion to this, the authors report methods consistent with a diagnostic test prediction model development and validation, which suggest that they only wanted to predict the risk of having a positive SARS-CoV-2 test. Furthermore, the authors mention throughout their title and rest of the manuscript that their model is meant to guide targeted testing in Qatar. In summary, it is not very clear if the authors wanted to 1. develop a prediction model for having a positive SARS-CoV-2 test (which does not necessarily imply having symptomatic disease and would not capture all individuals exposed to SARS-CoV-2), 2. a prediction model for the clinical diagnosis of COVID-19 as the name implies (this would have required at least considering symptoms), or 3. a prediction model of the risk of having been exposed to SARS-CoV-2. 

Even when the study has strengths like the large sample size and robust statistical analyses, there are important and serious limitations for this study which could even have important implications that would favor segregation and non-equality of people in Qatar: 

1) The authors seem to have used a very limited dataset which only allowed them to include a small set of predictors. Evaluating symptoms would have been a minimal requirement towards developing a tool to perform targeted testing since the WHO has used symptoms to define suspected COVID-19 cases from the start of the pandemic.

2) The predictors identified have no pathophysiological basis (or at least the authors have not adequately described this) for what they are intending to predict. It is very likely that these associations are instead spurious and only reflect confounding due to the characteristics of people who tend to seek diagnostic testing more often. For instance, sex has been included in the model, does this mean that women should not be prioritized for testing? As the authors commented on, their population is predominantly composed of men. The implications of this cannot be underestimated, since prioritizing men for testing would favor sex inequity and could limit the access of women to healthcare. The nationality variable could also only be reflecting the diversity of inhabitants in Qatar, reason why guiding testing according to nationality would also be unethical and could favor discrimination of foreigners in Qatar. It would be very irresponsible in my opinion to use this model to guide diagnostic testing in its current form and without having tested all other important predictors.

Please take these comments very seriously and re-consider if you still want to pursue publication of this paper. Otherwise, extensive modifications are needed since it could be very risky to publish it as it is and there would be a very high risk of the study being retracted due to these important concerns despite the journal where it ends up being published. I am quite sure that the authors had not thought of the implications that their findings could have, and they should not be blamed for it. 

The following comments could be useful to the authors to improve their manuscript:

1) The authors do not mention their study design (cross-sectional, cohort, experimental, etc.) at any moment in their methods. Labeling the study design is a current requirement by the Equator-Network recommendations. Note that diagnostic and prognostic prediction models can be derived from 1) cross-sectional 2) cohort, or 3) clinical trials, which is why it is important to fully report this study according to the corresponding study design.

2) The authors may need to review the STROBE statement (https://pubmed.ncbi.nlm.nih.gov/17941715/) and RECORD extension (https://pubmed.ncbi.nlm.nih.gov/26440803/) to report their study since it has characteristics of an observational study using routinely collected health data. Please provide the STROBE checklist + RECORD checklist as supplementary material for peer-review only: https://docs.google.com/viewer?url=http://www.record-statement.org/Files/checklist/RECORD%20Checklist.pdf

3) Additionally, since this study involved the development and validation of a prediction model, the authors would need to report their study according to TRIPOD recommendations (https://pubmed.ncbi.nlm.nih.gov/25560730/). Please provide the TRIPOD checklist as supplementary material for peer-review only to make sure that all elements for adequate reporting have been included.

4) Since the authors are claiming that their study is the first COVID-19 risk score, they need to perform a systematic search of the literature to be able to confidently conclude that. It is not acceptable to say "we believe that" without having made any serious attempts to address that uncertainty. This systematic search of the literature needs to be reported according to as many as possible of PRISMA-S recommendations (https://www.equator-network.org/reporting-guidelines/prisma-s/) even when this is not part of a systematic review per se.

5) The authors mention a prospective validation strategy "The second objective was to assess the prospective performance of this risk sore on epidemic data collected after its derivation", however, their methods are not compatible with having prospectively evaluated the performance of the model since they later declare that they split the sample into 50% for the development of the model and 50% for its validation, which clearly suggests that these analyses were retrospective. Please correct since this is misleading. 

5) In case that it was indeed prospective at any point, was this study registered in a publicly available website or was the research protocol made available for public scrutiny? This would have been important to foster transparency, especially since the authors describe a prospective validation component. Please read the following to see what I mean: https://doi.org/10.1371/journal.pmed.1001711

6) Please rename your score, since COVID-19 risk score is very non-specific. At least add "Qatar" to the name of the score in the title as you have done in some sections of the manuscript.

7) Page 5: Please be more specific on the strategy for sampling.