Write a comment

The manuscript presents an interesting elicitation method that allows the elicitation of the observable quantities to be accompanied by the elicitation of the uncertainty around them. Since quantile probability tuples (QPTs) are more natural to operate for humans than e.g. probability density functions, this work provides an excellent novel methodological opening how the elicitation method can infer based on QPTs provide by the expert.

The authors "describe and illustrate the hybrid elicitation approach for parameterizing the QDirichlet prior for describing the uncertainty in the {𝑝, 𝑞}𝑛 [quantile-probability-tuplets] parameters of the proper metalog". In simple words, the authors introduce an elicitation method for a Dirichlet (or Connor-Mosimann) prior in the context of meta-logistic distribution (Keelin, 2016) as a likelihood. In the world of quantile-parametrized distributions, their method can be regarded as a model-specific elicitation method for a meta-logistic model with a (Q)Dirichlet prior. 

The manuscript is, at some parts, difficult to place side by side with the already existing elicitation literature. The adopted terminology is not consistent with the literature. For instance, in the first paragraph the authors write “Elicitation of parameters” is “known as the parametric elicitation (Garthwaite et al., 2005; O’Hagan, 2019)”. Only the former of the referred works use the term “parametric elicitation”, and it’s used for a prior elicitation method that assume a parametric prior distribution (in contrast to non-parametric prior distribution). Similarly, in the second paragraph, the authors hint that the “predictive elicitation” is not about eliciting a prior distribution for model parameters but a distribution for “next observation”, but this not true in the referred work (Akbarov, 2009, p.121), not to mention the early research on this research line in prior elicitation (Kadane and Wolfson, 1998; Kadane et al., 1980; Oman, 1985; Garthwaite and Dickey, 1988; Ibrahim and Laud, 1994; Bedrick et al., 1996; Chen and Ibrahim, 2003; Denham and Mengersen, 2007; Elfadaly and Garthwaite, 2011; Garthwaite et al., 2013; Elfadaly and Garthwaite, 2015). 

I expect that the manuscript can evolve to an excellent paper, if (i) the connection to the existing elicitation literature is made clearer and the terminology is harmonized with that (a minimum requirement is that potential different usage of terms in references is explicitly stated), and (ii) the applicability of the proposed elicitation method is made more explicit (e.g. Is this elicitation method for any Dirichlet prior, or an elicitation method for a meta-logistic model with a QDirichlet prior, or both? How the method works with real human experts? etc..).