Parameter inference in dynamical systems with co-dimension 1 bifurcations

Dynamical systems with intricate behaviour are all-pervasive in biology. Many of the most interesting biological processes indicate the presence of bifurcations, i.e. phenomena where a small change in a system parameter causes qualitatively different behaviour. Bifurcation theory has become a rich field of research in its own right and evaluating the bifurcation behaviour of a given dynamical system can be challenging. An even greater challenge, however, is to learn the bifurcation structure of dynamical systems from data, where the precise model structure is not known. Here, we study one aspects of this problem: the practical implications that the presence of bifurcations has on our ability to infer model parameters and initial conditions from empirical data; we focus on the canonical co-dimension 1 bifurcations and provide a comprehensive analysis of how dynamics, and our ability to infer kinetic parameters are linked. The picture thus emerging is surprisingly nuanced and suggests that identification of the qualitative dynamics—the bifurcation diagram—should precede any attempt at inferring kinetic parameters.


Recommendation?
Accept with minor revision (please list in comments)

Comments to the Author(s)
The paper shows that parameter estimation is strongly affected by the class of dynamical system underlying the data. The authors focus in particular on simple one-parameter systems with bifurcations. The position of the system in the phase space of the model has a strong effect on parameter identifiability as quantified here by the Fisher Information. The Fisher information is often high close to bifurcation events because small parameter changes there can lead to large changes in the system behaviour and consequently a large effect on the log-likelihood of the model. The paper provides a systematic study of the Fisher information on the parameter space of different types of system exhibiting bifurcations.
I think the observations in the paper are interesting, as linking the dynamical properties of the system to parameter identifiability is of interest. The take-home message seems to be that one should consider the qualitative behaviour of the system prior to parameter estimation. However, in all but the simplest models the parameter space will be very large and therefore I'm not sure how easy it is to really map out the phase-space as is done for this simple one-dimensional system. Nevertheless, I think it is interesting to link model analysis with parameter estimation as is considered here. Previous studies looked simply at whether the parameters are inferable at all, whereas focussing on the Fisher information provides a more quantitative way to consider whether parameter estimation is feasible with finite data.
Major Comments: 1) Although there is no data I think it would be useful to provide the code to reproduce the figures in the paper. This should be easy to do and useful for people who wish to build on this work and get into the detail of the results.
2) The log-likelihood is often very far from normal or even symmetrical around the ML solution and therefore the Fisher Information doesn't fully capture the problem of parameter inference as it is really telling us about the asymptotic efficiency of parameter estimation. I think this limitation should be mentioned, that the Fisher Information is only using 2nd order information about the likelihood surface.
3) Related to this issue, in the context of parameter estimation in chaotic systems, for example, this does not help us much as the likelihood surface is very jagged. Therefore, this framework will not help understand every kind of dynamical system. Berliner (Journal of the American Statistical Association, 86, 1991) discusses the difficulty of interpreting the Fisher Information in such cases. 4) I think it should be pointed out that Bayesian parameter estimation would not require parameters to be identifiable in order to make useful inferences (e.g. about which region of the phase space we are in) and considerations about the Fisher Information are less relevant for Bayesian parameter inference. Nevertheless, I think in the Bayesian case it would still be useful to identify the phase structure of the model as one could ask the right questions, e.g. what is the posterior probability of the system being in a particular region of the phase space?
Minor comments P6. Make it clear this is the observed Fisher Information as there are two versions.

Review form: Reviewer 2
Is the manuscript scientifically sound in its present form? Yes

Do you have any ethical concerns with this paper? No
Have you any concerns about statistical analyses in this paper? No

Recommendation?
Accept with minor revision (please list in comments)

Comments to the Author(s)
This is a nice, short article that makes a useful contribution to an important mathematical subject that has widespread applications. The authors investigate parameter inference in a range of single-dimension, single-parameter dynamical systems with co-dimension 1 bifurcations. Multidimensional, multi-parameter systems that exhibit the same type of bifurcations are widespread; for example in molecular biology, ecology, and epidemiology. Parameter inference for these models is a subject of paramount importance that is still largely unsolved and therefore contributions, such as this article, are very useful.
The paper is easily readable and includes beautiful illustrations that are also fairly easy to comprehend.
I have some recommendations that I include below in order of significance: 1) There are a few surprising results that are not discussed thoroughly. For example, in supercritical pitchfork bifurcation when a>0, the Fisher information is much lower at the peaks and troughs than at the unstable fixed point. However, experimental observations are often taken at the peak times. Therefore, according to the results presented here, parameter inference will often rely on these low-information (in the sense of Fisher Information) observations. I suggest that the authors highlight this and similar results, and make sure the reasons behind these observations are clearly explained.
2) I wish the authors have given more information regarding the effects of noise and sparsity. The general conclusion is that the results are largely unchanged. However, I would suspect some breaking points both when the observations are too sparse (e.g. in supercritical pitchfork bifurcation with a>0, the observations are too sparse to observe the periodicity of the solution) or too noisy. It will be a substantial improvement if such breaking points are roughly given.
3) Some more details on the likelihood and the Fisher Information computation are necessary for the results to be replicable. For example, the time-points of the simulated observations. Providing the differential equations for the Fisher Information (in appendix) will also make the results more accessible. 4) The literature review for parameter inference on dynamical systems needs to be extended. 5) I suggest to clarify better the novelty compared to the most relevant publications (e.g. 22 and 23). 6) In section "Generating Simulated Data" where noise is discussed, I suggest a brief statement that much more detailed models of the stochasticity of dynamical systems are often necessary or useful as well as providing a few citations of relevant work. 6) There are a few typos that I'm sure the authors will correct in the next version.

09-Aug-2019
Dear Dr Stumpf, On behalf of the Editors, I am pleased to inform you that your Manuscript RSOS-190747 entitled "Parameter inference in dynamical systems with co-dimension 1 bifurcations" has been accepted for publication in Royal Society Open Science subject to minor revision in accordance with the referee suggestions. Please find the referees' comments at the end of this email.
The reviewers and handling editors have recommended publication, but also suggest some minor revisions to your manuscript. Therefore, I invite you to respond to the comments and revise your manuscript.
• Ethics statement If your study uses humans or animals please include details of the ethical approval received, including the name of the committee that granted approval. For human studies please also detail whether informed consent was obtained. For field studies on animals please include details of all permissions, licences and/or approvals granted to carry out the fieldwork.
• Data accessibility It is a condition of publication that all supporting data are made available either as supplementary information or preferably in a suitable permanent repository. The data accessibility section should state where the article's supporting data can be accessed. This section should also include details, where possible of where to access other relevant research materials such as statistical tools, protocols, software etc can be accessed. If the data has been deposited in an external repository this section should list the database, accession number and link to the DOI for all data from the article that has been made publicly available. Data sets that have been deposited in an external repository and have a DOI should also be appropriately cited in the manuscript and included in the reference list.
If you wish to submit your supporting data or code to Dryad (http://datadryad.org/), or modify your current submission to dryad, please use the following link: http://datadryad.org/submit?journalID=RSOS&manu=RSOS-190747 • Competing interests Please declare any financial or non-financial competing interests, or state that you have no competing interests.
• Authors' contributions All submissions, other than those with a single author, must include an Authors' Contributions section which individually lists the specific contribution of each author. The list of Authors should meet all of the following criteria; 1) substantial contributions to conception and design, or acquisition of data, or analysis and interpretation of data; 2) drafting the article or revising it critically for important intellectual content; and 3) final approval of the version to be published.
All contributors who do not meet all of these criteria should be included in the acknowledgements.
We suggest the following format: AB carried out the molecular lab work, participated in data analysis, carried out sequence alignments, participated in the design of the study and drafted the manuscript; CD carried out the statistical analyses; EF collected field data; GH conceived of the study, designed the study, coordinated the study and helped draft the manuscript. All authors gave final approval for publication.
• Acknowledgements Please acknowledge anyone who contributed to the study but did not meet the authorship criteria.
• Funding statement Please list the source of funding for each author.
Please ensure you have prepared your revision in accordance with the guidance at https://royalsociety.org/journals/authors/author-guidelines/ --please note that we cannot publish your manuscript without the end statements. We have included a screenshot example of the end statements for reference. If you feel that a given heading is not relevant to your paper, please nevertheless include the heading and explicitly state that it is not relevant to your work.
Because the schedule for publication is very tight, it is a condition of publication that you submit the revised version of your manuscript before 18-Aug-2019. Please note that the revision deadline will expire at 00.00am on this date. If you do not think you will be able to meet this date please let me know immediately.
To revise your manuscript, log into https://mc.manuscriptcentral.com/rsos and enter your Author Centre, where you will find your manuscript title listed under "Manuscripts with Decisions". Under "Actions," click on "Create a Revision." You will be unable to make your revisions on the originally submitted version of the manuscript. Instead, revise your manuscript and upload a new version through your Author Centre.
When submitting your revised manuscript, you will be able to respond to the comments made by the referees and upload a file "Response to Referees" in "Section 6 -File Upload". You can use this to document any changes you make to the original manuscript. In order to expedite the processing of the revised manuscript, please be as specific as possible in your response to the referees. We strongly recommend uploading two versions of your revised manuscript: 1) Identifying all the changes that have been made (for instance, in coloured highlight, in bold text, or tracked changes); 2) A 'clean' version of the new manuscript that incorporates the changes made, but does not highlight them.
When uploading your revised files please make sure that you have: 1) A text file of the manuscript (tex, txt, rtf, docx or doc), references, tables (including captions) and figure captions. Do not upload a PDF as your "Main Document"; 2) A separate electronic file of each figure (EPS or print-quality PDF preferred (either format should be produced directly from original creation package), or original software format); 3) Included a 100 word media summary of your paper when requested at submission. Please ensure you have entered correct contact details (email, institution and telephone) in your user account; 4) Included the raw data to support the claims made in your paper. You can either include your data as electronic supplementary material or upload to a repository and include the relevant doi within your manuscript. Make sure it is clear in your data accessibility statement how the data can be accessed; 5) All supplementary materials accompanying an accepted article will be treated as in their final form. Note that the Royal Society will neither edit nor typeset supplementary material and it will be hosted as provided. Please ensure that the supplementary material includes the paper details where possible (authors, article title, journal name).
Supplementary files will be published alongside the paper on the journal website and posted on the online figshare repository (https://rs.figshare.com/). The heading and legend provided for each supplementary file during the submission process will be used to create the figshare page, so please ensure these are accurate and informative so that your files can be found in searches. Files on figshare will be made available approximately one week before the accompanying article so that the supplementary material can be attributed a unique DOI.
Please note that Royal Society Open Science charge article processing charges for all new submissions that are accepted for publication. Charges will also apply to papers transferred to Royal Society Open Science from other Royal Society Publishing journals, as well as papers submitted as part of our collaboration with the Royal Society of Chemistry (http://rsos.royalsocietypublishing.org/chemistry).
If your manuscript is newly submitted and subsequently accepted for publication, you will be asked to pay the article processing charge, unless you request a waiver and this is approved by Royal Society Publishing. You can find out more about the charges at http://rsos.royalsocietypublishing.org/page/charges. Should you have any queries, please contact openscience@royalsociety.org.
Once again, thank you for submitting your manuscript to Royal Society Open Science and I look forward to receiving your revision. If you have any questions at all, please do not hesitate to get in touch. The paper shows that parameter estimation is strongly affected by the class of dynamical system underlying the data. The authors focus in particular on simple one-parameter systems with bifurcations. The position of the system in the phase space of the model has a strong effect on parameter identifiability as quantified here by the Fisher Information. The Fisher information is often high close to bifurcation events because small parameter changes there can lead to large changes in the system behaviour and consequently a large effect on the log-likelihood of the model. The paper provides a systematic study of the Fisher information on the parameter space of different types of system exhibiting bifurcations.
I think the observations in the paper are interesting, as linking the dynamical properties of the system to parameter identifiability is of interest. The take-home message seems to be that one should consider the qualitative behaviour of the system prior to parameter estimation. However, in all but the simplest models the parameter space will be very large and therefore I'm not sure how easy it is to really map out the phase-space as is done for this simple one-dimensional system. Nevertheless, I think it is interesting to link model analysis with parameter estimation as is considered here. Previous studies looked simply at whether the parameters are inferable at all, whereas focussing on the Fisher information provides a more quantitative way to consider whether parameter estimation is feasible with finite data.
Major Comments: 1) Although there is no data I think it would be useful to provide the code to reproduce the figures in the paper. This should be easy to do and useful for people who wish to build on this work and get into the detail of the results.
2) The log-likelihood is often very far from normal or even symmetrical around the ML solution and therefore the Fisher Information doesn't fully capture the problem of parameter inference as it is really telling us about the asymptotic efficiency of parameter estimation. I think this limitation should be mentioned, that the Fisher Information is only using 2nd order information about the likelihood surface.
3) Related to this issue, in the context of parameter estimation in chaotic systems, for example, this does not help us much as the likelihood surface is very jagged. Therefore, this framework will not help understand every kind of dynamical system. Berliner (Journal of the American Statistical Association, 86, 1991) discusses the difficulty of interpreting the Fisher Information in such cases. 4) I think it should be pointed out that Bayesian parameter estimation would not require parameters to be identifiable in order to make useful inferences (e.g. about which region of the phase space we are in) and considerations about the Fisher Information are less relevant for Bayesian parameter inference. Nevertheless, I think in the Bayesian case it would still be useful to identify the phase structure of the model as one could ask the right questions, e.g. what is the posterior probability of the system being in a particular region of the phase space?
Minor comments P6. Make it clear this is the observed Fisher Information as there are two versions.
P6. After (2.9) "we focussed on theta = (0, 0.1)" but alpha is studied away from zero, I didn't understand this. Typos: P5. "much better is than a" P6. After (2.8) "carry about about" P7. "sround alpha = 0" P8. "high-values of the Fisher Information" should not be hyphenated P10. Figure 5 caption "margnialised" P12. "pakcage" Reviewer: 2 Comments to the Author(s) This is a nice, short article that makes a useful contribution to an important mathematical subject that has widespread applications. The authors investigate parameter inference in a range of single-dimension, single-parameter dynamical systems with co-dimension 1 bifurcations. Multidimensional, multi-parameter systems that exhibit the same type of bifurcations are widespread; for example in molecular biology, ecology, and epidemiology. Parameter inference for these models is a subject of paramount importance that is still largely unsolved and therefore contributions, such as this article, are very useful.
The paper is easily readable and includes beautiful illustrations that are also fairly easy to comprehend.
I have some recommendations that I include below in order of significance: 1) There are a few surprising results that are not discussed thoroughly. For example, in supercritical pitchfork bifurcation when a>0, the Fisher information is much lower at the peaks and troughs than at the unstable fixed point. However, experimental observations are often taken at the peak times. Therefore, according to the results presented here, parameter inference will often rely on these low-information (in the sense of Fisher Information) observations. I suggest that the authors highlight this and similar results, and make sure the reasons behind these observations are clearly explained. 2) I wish the authors have given more information regarding the effects of noise and sparsity. The general conclusion is that the results are largely unchanged. However, I would suspect some breaking points both when the observations are too sparse (e.g. in supercritical pitchfork bifurcation with a>0, the observations are too sparse to observe the periodicity of the solution) or too noisy. It will be a substantial improvement if such breaking points are roughly given.
3) Some more details on the likelihood and the Fisher Information computation are necessary for the results to be replicable. For example, the time-points of the simulated observations. Providing the differential equations for the Fisher Information (in appendix) will also make the results more accessible. 4) The literature review for parameter inference on dynamical systems needs to be extended. 5) I suggest to clarify better the novelty compared to the most relevant publications (e.g. 22 and 23). 6) In section "Generating Simulated Data" where noise is discussed, I suggest a brief statement that much more detailed models of the stochasticity of dynamical systems are often necessary or useful as well as providing a few citations of relevant work. 6) There are a few typos that I'm sure the authors will correct in the next version.

20-Sep-2019
Dear Dr Stumpf, I am pleased to inform you that your manuscript entitled "Parameter inference in dynamical systems with co-dimension 1 bifurcations" is now accepted for publication in Royal Society Open Science.
You can expect to receive a proof of your article in the near future. Please contact the editorial office (openscience_proofs@royalsociety.org and openscience@royalsociety.org) to let us know if you are likely to be away from e-mail contact --if you are going to be away, please nominate a coauthor (if available) to manage the proofing process, and ensure they are copied into your email to the journal.
Due to rapid publication and an extremely tight schedule, if comments are not received, your paper may experience a delay in publication.
Royal Society Open Science operates under a continuous publication model (http://bit.ly/cpFAQ). Your article will be published straight into the next open issue and this will be the final version of the paper. As such, it can be cited immediately by other researchers. As the issue version of your paper will be the only version to be published I would advise you to check your proofs thoroughly as changes cannot be made once the paper is published.
On behalf of the Editors of Royal Society Open Science, we look forward to your continued contributions to the Journal.

Kind regards, Andrew Dunn Royal Society Open Science Editorial Office
We are grateful for the reviewers' positive assessment of our work, and we are delighted how their comments have allowed us to improve our presentation, and hopefully reach a larger audience more effectively.

Reviewer 1
Major Comments: Comment 1: We have modified the paragraph "DataAccessibility". It says now: "All calculations were carried out in the Julia language. The code is open source and available on https://github.com/LislPisl/ Bifurcations." Comment 2: We regard the Fisher information as a useful but imperfect tool to quantify uncertainty around MLE (or MAP estimate). The limitations of the Fisher Information are now referred to in two places and we have provided some additional references regarding the use and scope of such measures.

Comment 3:
We agree entirely and have amended our discussion in light of the reviewers comments. We would also like to thank them for the Berliner reference which we had not previously come across but which is proving useful in our ongoing work.
Comment 4: This is a very perceptive comment (and one of the authors has argued along similar lines in the past). Again this (identifying the phase structure of dynamical systems) is part of further work, but we have added a brief Bayesian perspective.

Minor Comments:
We have done this in the revised manuscript.
Reviewer 2 Comment 1: We had previously tried to avoid repetition but can see how this would have made some of the results harder to follow. We have now extended our discussion and elaborated on our observations and what can be learned from them.

Comment 2:
We fully unterstand this point. The interplay between noise and dynamics is, however, subtle. For example, some bifurcations are masked or even fully destroyed in stochastic dynamical counterparts to deterministic systems exhibiting them. We have therefore tried to focus on observational noise and experimental resolutions. Fearing that some of these results might appear obvious, if only by hindsight, we had previously opted for brevity but as the reviewer's comments make clear, our discussion is now reorganised and expanded.

Comment 3:
We are now sharing a Julia package which will enable people to reproduce our results. As the Julia syntax is closely aligned with that of other mathematical programming languages we are expecting that typical Mathematica/Matlab/Python users would straightforwardly be able to adapt the code for their purposes; or even better, explore Julia.

Comment 4:
We have done this in the revision.
Comment 5: Our revised manuscript does this much more clearly.