PREreview of Heterogeneity in ligand-bound TRPV1: A comparison of methods in cryo-EM and molecular dynamics simulation

Summary

It is well established that proteins exist as an ensemble of conformations, however, it is typical for most solved structures (either by Cryo-EM or X-ray crystallography) to be represented as a single conformation representative of the most populated signals in the ensemble averaged data. Identifying alternative conformations is not only necessary for understanding function but these conformations can also be targeted by therapeutics. However, these states often occupy minor populations within a data set that get obscured by either noise. In this work, the authors demonstrate the utility of several available methods to study both compositional and conformational heterogeneity present in a publicly available single particle Cryo-EM dataset of TRPV1 by conducting a comparative analysis of five methods that reveal both conformational and compositional heterogeneity. The authors demonstrate that each method is sensitive to distinct aspects of this variability, however, not all methods can accurately discern all aspects of heterogeneity in the dataset. Further, each method is implemented differently which makes direct comparisons between methods non-trivial. To address this, the study also introduces tools like Bayesian ensemble reweighting and CryoSVD to integrate and compare results across methods, providing a framework for systematic evaluation. The major success of this paper lies in being the first comprehensive comparison of multiple cryo-EM heterogeneity analysis methods, revealing both the strengths of each method and unique insights into the conformational landscape of TRPV1. In addition, the authors demonstrate that compositional heterogeneity in single particle Cryo-EM datasets can be detected with small subsets of data that may otherwise be overlooked. This runs counter to the expectation that minorly populated states within an ensemble can only be accurately resolved by increasing the number of particles used. In particular, the authors use of Bayesian ensemble reweighting using MD simulations of TRPV1 reveal that even rare conformational changes can be detected using a small subset of a full particle stack that may otherwise be overlooked by existing methods for modeling heterogeneity, albeit, they can still be influenced by the choice of starting conformations used in the ensemble. This work establishes a framework for evaluating structural heterogeneity across different methodologies. The major weakness of the paper is its unclear framing of its primary goal: Is it primarily aimed at resolving new insights about TRPV1, or is TRPV1 being used as a representative system to benchmark the analysis methods? This ambiguity makes it difficult to fully assess the scope and focus of the study. Overall, this work paves the way for future work aiming to do larger scale benchmarking experiments.

Major Points

• Clarify the Overall Message of the Paper The goals of the paper can benefit from clearer articulation. Is the primary aim to provide new biological insights about TRPV1, or is TRPV1 primarily being used as a case study to evaluate and benchmark heterogeneity analysis methods?

  • If the paper focuses on TRPV1, it would benefit from additional detail about why TRPV1 is biologically significant, how its conformational heterogeneity relates to its function, and how the findings advance our understanding of this protein. For example, consider elaborating on the relevance of TRPV1’s open and closed states to its physiological role.

  • We think that on page 14, the statement that “While we believe other datasets with fewer complicating factors… are better suited to benchmarking and methods development, the results presented here serve as a solid introduction to the ways in which resolving heterogeneity from single-particle cryo-EM datasets can be complicated by a variety of factors, such that the choice of method used to study a dataset will heavily influence the kinds of heterogeneity a user will resolve” clarifies the paper’s aims. However, this point should be made earlier in the paper to set clearer expectations. Additionally, it would be helpful to explain why TRPV1 was chosen as the case study if it is not ideally suited for benchmarking, considering the aim seems to be related to benchmarking.

• Contextualizing Results You state that each heterogeneity method is best suited to specific use cases (e.g., conformational vs. compositional heterogeneity) and that the insights into TRPV1 come from combining these methods. However, this raises practical questions for the field:

  • If heterogeneity type (conformational or compositional) cannot be known before analysis, how should researchers decide which method(s) to apply? Should they always apply multiple methods?

  • The authors point out that when using 3Dflex: “We found that by independently training two models with the same hyperparameters we could obtain noticeably different reconstructions and learned deformations. Despite these noticeable differences, these models yield the same training scores and global resolutions” This seems like it would be problematic. How can you be certain that the conformational changes derived are true and not the result of fitting into noise? Further, as mentioned when using cryoDRGN, cryoDRGN could only detect the compositional heterogeneity present when DkTx is either bound or unsound but did not detect conformational changes. If the objective of this paper is to illustrate how these different methods for resolving heterogeneity can be used then how do you resolve the absence of agreement between two methods?

Minor points:

• Disagreement Between Analysis Methods

The results from MD simulations are stated to reveal motions similar to those found using cryo-EM heterogeneity analysis tools. However, there appears to be a discrepancy in the population distributions described:

• On page 4, you note that “each method consistently indicated the presence of a population of particles occupying states closer to the apo closed state, rather than the ligand-bound open state...”

• On page 11, you state, “reweighting the MD distribution to that of the cryo-EM data indicates that the majority of particles occupy a conformation close to the ligand-bound open state, but with a small population of particles closer to the apo closed state.”

• This discrepancy is not addressed in the manuscript. What is the takeaway here? These results do not appear to be consistent with one another, so how should the reader interpret them? A brief discussion reconciling these findings or clarifying how method-specific biases might explain the differences would be helpful.

• Validation and Hallucination Issues in Heterogeneity Methods

You note that heterogeneity analysis methods can suffer from hallucination issues, where spurious conformations might be inferred due to noise or limitations of the algorithm. However, it is unclear how you assess whether the conformations detected in this study are real.

• Testing on synthetic data is a common way to validate such methods, as it provides a ground truth for comparison. You briefly mention the existence of available synthetic data on page 3 (citation 13). Consider expanding on why you chose not to perform this analysis using that dataset, or one similar to it. This ties back to the need to justify the choice of TRPV1 as a case study, as synthetic datasets could provide stronger validation of the methods’ performance.

• “The extra density of the toxin may cause artifacts in the volume-preserving flow field”

  • Is this to mean that the receptor model is trying to fit into this density by flexing (but without considering the ligand being present (Thus not properly handling both conformational and compositional heterogeneity simultaneously)? How do you reconcile this with the observations made in figure 7 that ensemble reweighting shows that most of the particles occupy the apo-state? If the particles are actually apo for the most part, wouldn’t that affect the interpretation of the 3Dflex results?

•  “Indeed, we notice that when structures from all three temperatures at which we ran simulations (150 K, 200 K, and 310 K) are used in reweighting, the structures derived from lower temperature simulations received higher weights (Supplementary Figure S3B-C, Supplementary Figure S4). Indicating that the sample ensemble may be significantly cooler than the temperature the sample’s initial temperature 277 K (4°). These results show that ensemble reweighting is able to detect rare states in this sample, despite the fact that the simulations are carried out using classical forcefields at different temperatures from the incubated temperature of the sample using.”

  • This seems to have some grammatical errors needs to be revised to be clearer.

• Figure 5D is missing a caption description. 

• In the main text of the paper, you should reference the relevant methods sections. For example, in section 2.7, you introduce the concept of power spectrums without context. Referencing methods section 5.4 would allow the reader to know they must read that section to gain a more complete understanding of the results you present. 

Competing interests

The authors declare that they have no competing interests.