PREreview of Interpretable Biomanufacturing Process Risk and Sensitivity Analyses for Quality-by-Design and Stability Control

Formal Peer Review

Manuscript Title: Interpretable Biomanufacturing Process Risk and Sensitivity Analyses for Quality-by-Design and Stability Control

Submitted to: Journal of Pharmaceutical Sciences / Biotechnology and Bioengineering

Review Status: Major Revisions Required

1. Summary Assessment

This manuscript presents a methodologically substantive framework integrating Bayesian network (BN)-based probabilistic knowledge graphs with Shapley value (SV)-derived sensitivity analyses to address CPP/CQA criticality assessment and model uncertainty quantification in biomanufacturing. The three-part contribution — interpretable process modelling, stochastic sensitivity analysis, and model risk decomposition via a nested Gibbs sampler — represents a technically ambitious undertaking that meaningfully distinguishes itself from conventional multivariate PAT approaches by explicitly incorporating causal process structure.

The motivational framing around QbD, process stability, and the limitations of data-sparse biomanufacturing environments is well-constructed and professionally grounded in ICH Q8(R2) and FDA PAT guidance. The use of game-theoretic sensitivity analysis (Owen, 2014; Song et al., 2016) to handle interdependent inputs in complex bioprocesses is novel and conceptually sound. The inclusion of both a simulation study and a real-world Yarrowia lipolytica citric acid fermentation case study strengthens empirical credibility.

However, the manuscript exhibits several significant deficiencies that limit its publication readiness in its current form. These include: the restriction to linear Gaussian BN models without adequate justification or sensitivity testing for nonlinear bioprocesses; an extremely small real-world dataset (R = 8 batches) that undermines inferential reliability; inadequate discussion of graph structure determination and its sensitivity; underdeveloped regulatory alignment; and, critically, an absence of formal statistical validation metrics for the real case study. Additionally, the manuscript appears to have been posted as an arXiv preprint (arXiv:1909.04261v4) since 2019, and the authors should disclose whether this constitutes a simultaneous or prior submission.

2. Major Concerns

MC1 – Linearity Assumption and Model Validity The entire analytical framework rests on a linear Gaussian BN specified in Equations (4)–(6). Biomanufacturing processes are inherently nonlinear, as evidenced by the exponential cell growth kinetics the authors themselves cite (Section 4). While a log-transformation is proposed to linearise the exponential growth model, this approach is applied only illustratively and is not generalised to the broader framework. The authors must provide: (a) a formal analysis of when the linear approximation is adequate; (b) empirical evidence that the linearization does not introduce systematic bias in the real case study; and (c) a discussion of how the framework degrades when linearity assumptions are violated. The claim that finite-difference approximation of ODE-based kinetics matches Equation (6) requires rigorous justification, not merely assertion.

MC2 – Critically Insufficient Real-World Sample Size The real case study employs only R = 8 complete batches to estimate a BN with 62 nodes. The posterior standard deviations reported in Tables 5–7 are frequently comparable to or exceed the estimated criticality values themselves (e.g., Cell_0 contributing 1.88% ± 4.12% to BM_10 variance), indicating that the posterior is effectively uninformative for many parameters. The authors acknowledge this limitation without adequately addressing its implications. With 62 nodes, the parameter space includes hundreds of coefficients, making credible posterior inference with 8 observations statistically untenable without strong, explicitly validated prior information. The authors must either: (a) formally quantify the minimum sample size required for reliable inference under the proposed framework; or (b) provide a more rigorous justification for the informativeness of the priors used, including sensitivity analysis to prior hyperparameter selection.

MC3 – BN Structure Determination Not Addressed. The manuscript assumes that the DAG structure of the BN is known a priori. This is a fundamental methodological gap. In practice, causal graph structure is rarely fully known, and structure uncertainty can propagate substantially into criticality estimates. The authors must address: how the graph topology is determined in practice; what procedures are recommended for structure learning or expert elicitation; and how structural misspecification affects the SV-based criticality assessments. The absence of any structure uncertainty analysis is a material omission for a framework claiming to support QbD process development.

MC4 – Absence of Formal Validation Metrics in Real Case Study. Section 7.2 presents criticality estimates but provides no ground-truth comparison, no cross-validation, no predictive performance metric, and no benchmarking against existing methods. Unlike Section 7.1.2, which compares BN-SV against ML-M using simulated true parameters, the real case study offers no analogous validation. Statements such as "matches well with the data in Fig. 6" and "consistent with the data" constitute qualitative corroboration, not scientific validation. A held-out prediction exercise, leave-one-out cross-validation, or comparison with established fermentation knowledge should be provided.

MC5 – Regulatory Alignment Is Superficial. The manuscript invokes QbD, ICH Q8(R2), and PAT guidance but does not engage substantively with the regulatory implications of the proposed framework. Key questions left unaddressed include: How would this framework be presented in a regulatory submission (e.g., BLA, NDA)? What constitutes an acceptable level of model uncertainty for regulatory purposes? How does the BN-SV criticality metric relate to the FMEA-based risk ranking tools currently used in industry (e.g., risk priority numbers, severity-probability matrices as per ICH Q9)? How would DAG structure assumptions be validated for regulatory acceptance? For a journal targeting pharmaceutical sciences, these questions are not peripheral — they are central to demonstrating practical impact.

MC6 – Computational Scalability Not Addressed Algorithm 2 employs a nested Gibbs sampling scheme with parameters Nπ, BO, and BI that are set to 500, 5, and 20, respectively, based on a citation to Song et al. (2016). For a process graph with 62 nodes and hundreds of parameters, the computational cost of this nested scheme is potentially prohibitive in industrial settings. No analysis of computational complexity, wall-clock runtime, or scalability to larger process graphs is provided. This omission is particularly concerning given the manuscript's claim to support real-time monitoring and release.

3. Minor Concerns

mC1 – Typographical and Notation Inconsistencies: The manuscript contains several notation inconsistencies. In Table 8, the caption references "pFeed_20,CA_140" while the table body reports results for "rOil_28" and "Feed_23." This appears to be a labelling error that must be corrected. Additionally, the subscript "∗" notation used throughout Section 6 to denote posterior-based quantities is introduced informally and should be defined with greater precision.

mC2 – Figures Require Enhancement: Figures 3 and 7 are central to the manuscript's contribution but are difficult to read at print resolution. Node labels in Fig. 3 are particularly small. All figures should be provided at a minimum of 300 dpi with legible fonts. Figure 8 (Appendix A) is complex but lacks a concise explanatory caption.

mC3 – Abstract Omits Key Limitations: The abstract presents the framework without any acknowledgement of its assumptions or limitations. Journal standards for this venue require that abstracts reflect the actual scope and boundary conditions of the contribution.

mC4 – Related Work on Nonlinear BN and Dynamic Models Is Absent: The background section does not review dynamic BN (DBN) models, which are directly relevant to time-series bioprocess data as encountered in the real case study. Similarly, nonlinear extensions of BN sensitivity analysis (e.g., based on copula approaches or non-Gaussian distributions) are not discussed, making the literature review incomplete.

mC5 – Gibbs Sampler Convergence Diagnostics Not Reported: The Gibbs sampling procedure specifies T0 = 500 as burn-in and h = 10 as thinning, but provides no MCMC convergence diagnostics (e.g., Gelman-Rubin Rˆ statistic, trace plots, effective sample size). This is a methodological requirement for any Bayesian computation paper targeting a rigorous venue.

mC6 – Prior Hyperparameter Sensitivity: The prior distributions in Equation (13) are described as "vague", but the specific hyperparameter values used are not reported in the main text (only in Appendix E for the simulation study). For the real case study, the prior specification is entirely opaque. A sensitivity analysis to prior choice, or at a minimum a full disclosure of hyperparameters used, is required.

mC7 – Citation Formatting: Several citations appear inconsistently formatted. "Guideline, I. H. T. et al. (2009)" is an unusual citation format for ICH Q8(R2) and should reference the document directly. The Mitchell (2013) citation references a trade publication (BioPharm International) for the cause-and-effect matrix; this should be supplemented with a peer-reviewed source where possible.

4. Methodology Critique

The methodological architecture of this paper is its primary strength and, simultaneously, its primary source of vulnerability. The integration of BN-based causal modelling with Shapley-value decomposition is mathematically coherent, and the derivations in Appendices B and C are correctly executed under the stated assumptions. The nested Gibbs sampler for BN-SV-MU analysis is a technically sophisticated contribution that, to the reviewer's knowledge, represents a novel combination in the biomanufacturing context.

However, the methodological case rests critically on the linear Gaussian assumption (Equations 4–6), which the authors justify primarily through the Central Limit Theorem and an appeal to existing biopharmaceutical literature (Coleman and Block, 2006). This justification is insufficient. The CLT argument applies to the distribution of sums of random effects, not necessarily to individual process factors, many of which follow bounded, skewed, or multimodal distributions (e.g., impurity concentrations bounded above zero, cell viability measurements bounded between 0% and 100%). The practical consequence of this assumption should be empirically examined, not assumed.

The BN graph structure is treated as given throughout. In the simulation study, the true structure is used directly; in the real case study, the structure is presumably elicited from domain knowledge, but this process is not described. Graph structure misspecification can lead to systematically biased criticality estimates, and this is a well-known problem in the BN literature (Koller and Friedman, 2009, which the authors cite). The failure to address structure uncertainty while simultaneously claiming the framework supports reliable QbD decision-making represents a gap between the methodological assumptions and the practical claims.

The comparison with ML-M (Table 3) is informative but limited. The BN-SV approach outperforms ML-M primarily because it exploits the true causal graph structure — a significant advantage in a simulation where that structure is known. The comparison would be more informative if it included cases where the graph structure is misspecified, or if it benchmarked against more competitive methods such as regularised structural equation models or dynamic linear models.

Reproducibility is partially addressed through algorithmic descriptions (Algorithms 1–3) and the simulation data specification (Appendix E), but the real-world fermentation data are not made available, and no data availability statement is provided.

5. Data Presentation Evaluation

Tables 1, 2, 5, 6, 7: These tables convey criticality estimates and associated standard deviations across a large number of factor-output combinations. While comprehensive, they are difficult to parse in their current form. The authors should consider: (a) highlighting rows/columns with criticality exceeding a meaningful threshold (e.g., >5% or >10%); (b) using heat-map formatting or shading to aid interpretation; (c) reducing table density by moving lower-priority entries to supplementary material. The juxtaposition of mean and SD in a single cell (e.g., "1.88(4.12)") is functional but not ideal; a consistent notation should be defined explicitly and placed prominently.

Table 3: The comparison table is appropriately structured, though the MAE column values should be accompanied by a statistical significance test or confidence interval to demonstrate whether the BN-SV improvement over ML-M is significant, particularly for lower-criticality factors where the differences are marginal.

Tables 4 and 8: These model-uncertainty decomposition tables are among the most novel contributions of the paper and deserve greater discussion than they receive. The dominance of variance coefficients (v²₄ contributing 73.75% in Table 4) is an important finding with direct implications for data collection strategy, but its interpretation is underexplored.

Figures 3, 5, 7: The visualisation of criticality and model uncertainty through node darkness and edge weight is conceptually appealing, but the grayscale encoding is not perceptually linear, and the figures would benefit from a formal colourmap legend with quantitative thresholds. Figure 7 (the BN for the real case study) shows a complex 62-node structure, but it is rendered too small to be analytically useful.

Figure 6: The fermentation time-series data are presented adequately, though adding confidence bands or batch-to-batch variability overlays would strengthen the empirical context.

6. Contribution and Novelty Assessment

The manuscript's contribution is moderate to substantial within the computational bioprocess modelling domain, and moderate within the broader pharmaceutical sciences and GMP compliance literature.

The combination of BN-based probabilistic knowledge graphs with Shapley-value global sensitivity analysis for end-to-end biomanufacturing is novel as an integrated framework. The extension to model uncertainty decomposition via nested posterior Shapley analysis (BN-SV-MU) is the most technically original contribution and has potential significance beyond biomanufacturing, including applications in pharmaceutical process development more broadly.

The practical relevance is genuine: the framework addresses a recognised industry need for interpretable, data-efficient process risk tools that are compatible with QbD principles. The framework's modularity — its capacity to incorporate unit-operation-specific mechanistic priors — is particularly well-suited to the heterogeneous data environments of biopharmaceutical manufacturing.

However, the contribution is weakened by the linearity assumption, the unaddressed structure learning problem, and the inability to demonstrate reliable inference with realistic batch sizes. The framework, as presented, is more a proof-of-concept than a deployment-ready tool, and the discussion does not adequately distinguish between these characterisations.

7. Publication Suitability Scores

Dimension Score (1–10) Originality 7 Methodological Rigor 5 Practical Relevance 6 Clarity of Writing 6 Overall Publication Readiness 5

8. Editorial Recommendation

Major Revisions

The manuscript presents a technically interesting and potentially impactful framework, but requires substantial revision before it is suitable for publication. The core methodological concerns — particularly the linearity assumption, the sample size limitations for the real case study, the absence of structure uncertainty analysis, the lack of MCMC diagnostics, and the superficial regulatory engagement — must be addressed. The revision should also strengthen the real case study with formal validation metrics and provide greater transparency regarding prior specifications and computational costs.

The authors are encouraged to resubmit following a comprehensive revision. The reviewer does not recommend rejection, as the foundational framework is scientifically sound and the contributions are meaningful when properly scoped and validated.

9. Revision Roadmap

Priority 1 — Methodological Foundations (Required for Acceptance)

1. Provide a formal analysis of the conditions under which the linear Gaussian approximation is acceptable for biomanufacturing CPP/CQA distributions. Include simulation experiments testing framework performance under nonlinear ground-truth models.

2. Address BN structure uncertainty explicitly. Describe and demonstrate a procedure (expert elicitation protocol, structure learning algorithm, or sensitivity analysis to structural assumptions) that would be applied in practice.

3. Report full MCMC convergence diagnostics for all Gibbs sampling procedures in both the simulation and real case studies (Gelman-Rubin statistics, effective sample sizes, trace plots provided at minimum as supplementary material).

4. Specify and report all prior hyperparameters used in the real case study, and provide a prior sensitivity analysis demonstrating robustness to hyperparameter choice.

Priority 2 — Empirical Validation (Required for Acceptance)

1. Add formal validation analysis for Section 7.2: include leave-one-out cross-validation of predictive performance, or a held-out test batch, alongside quantitative comparison to at least one competing method (e.g., PLS-based criticality assessment, LASSO regression).

2. Discuss explicitly the inferential limitations imposed by R = 8 batches in the context of a 62-node model, and provide a minimum sample size recommendation grounded in the framework's own uncertainty quantification outputs.

Priority 3 — Regulatory and Industry Alignment (Strongly Recommended)

1. Add a dedicated subsection discussing how the framework aligns with or complements ICH Q9 risk management tools (FMEA, FMECA, cause-and-effect matrices), and how BN-SV criticality estimates would be interpreted and presented in a regulatory submission context.

2. Address how graph structure assumptions and model linearity could be validated to meet regulatory expectations for process understanding.

Priority 4 — Computational Transparency (Recommended)

1. Provide runtime benchmarks for Algorithms 1–3 at the scales used in both case studies, and discuss scalability to larger process graphs (e.g., 100+ nodes) relevant to full end-to-end supply chain models.

2. Discuss the sensitivity of the framework to the sampling parameters (Nπ, BO, BI) beyond the brief reference to Song et al. (2016).

Priority 5 — Presentation (Required Before Final Acceptance)

1. Correct the labelling error in the caption of Table 8.

2. Improve resolution and legibility of all figures, particularly Figures 3 and 7.

3. Revise the abstract to include a concise statement of key assumptions and limitations.

4. Add a data availability statement clarifying the status of the real-world fermentation dataset.

5. Disclose the arXiv preprint status and confirm compliance with the target journal's prior publication policy.

Competing interests

The authors declare that they have no competing interests.

Use of Artificial Intelligence (AI)

The authors declare that they did not use generative AI to come up with new ideas for their review.