Robust finite automata in stochastic chemical reaction networks

Finite-state automata (FSA) are simple computational devices that can nevertheless illustrate interesting behaviours. We propose that FSA can be employed as control circuits for engineered stochastic biological and biomolecular systems. We present an implementation of FSA using counts of chemical species in the range of hundreds to thousands, which is relevant for the counts of many key molecules such as mRNAs in prokaryotic cells. The challenge here is to ensure a robust representation of the current state in the face of stochastic noise. We achieve this by using a multistable approximate majority algorithm to stabilize and store the current state of the system. Arbitrary finite state machines can thus be compiled into robust stochastic chemical automata. We present two variants: one that consumes its input signals to initiate state transitions and one that does not. We characterize the state change dynamics of these systems and demonstrate their application to solve the four-bit binary square root problem. Our work lays the foundation for the use of chemical automata as control circuits in bioengineered systems and biorobotics.


Comments to the Author(s)
In the paper, the authors demonstrated the principles and modules needed for building finite state machines with chemical reaction networks. The paper is nicely written and is advised to be published with following suggestions: The authors first introduced the major logic that enables the implementation of finite state machines -the approximate majority algorithm, where the purpose of such an algorithm is to allow the system to converge to a single state. While the principle works flawlessly in-silico, one minor comment here, as this is a computational paper, is the difficulty in actual implementation with chemical or biochemical molecules. One may need to think carefully about how to design reversible molecules that would react following the proposed rules. It would thus help if the authors may provide some thoughts on the implementation if possible.
The second key module the authors implemented for building finite state machines is the transition mapping and introduction of "flip species". The authors demonstrated how the introduction of new species stabilizes the state transitions of finite state machines by showing the two cases of simulation with and without the implementation of "flip species" -with the noninclusion case erring with failed state transition. While this is a straightforward example, the stochasticity of the system may render this result difficult to reproduce. It would thus greatly help readers to visualize the noise and show the stabilization effect for running the two designs with multiple runs, then use the derived mean and standard deviation to visualize stability effect of flip species introduction provides, thus, a major comment here would be the introduction of such a figure added in figure 3. Also, it is not clear how the simulations of the non-flip-speciescases were conducted, this is essential to include for readers' reference.
In the following sections, the authors performed detailed parametric studies and characterization to learn how different rates affect the state-switching robustness of the finite state machines. The heatmaps offer clear visualization to learn the trend and the authors also provide thoughtful interpretations. However, another major comment here is that how the authors are evaluating the FSM correctness is unclear, and should be carefully explained. It is clear to interpret 100% and 0% correctness, but there are multiple regions with values lying in-between and is confusing for readers -for example, did the FSMs skip some states or did they fail to skip states? The authors should clarify the definition in the main text or in the captions below.
Finally, the implementation of solving the square root problem is a great and interesting example demonstrating the power of useful finite state machines with CRN implementations!

Review form: Reviewer 2
Is the manuscript scientifically sound in its present form? Yes

Are the interpretations and conclusions justified by the results? Yes
Is the language acceptable? Yes

Recommendation?
Accept as is

Comments to the Author(s)
The authors present a scheme to compile arbitrary Finite State Automata (FSA) into stochastic Chemical Reaction networks (CRN). They do this by using Approximate Majority to stabilize the state of the system after partial transitions are triggered by an Input. This makes the construction robust to stochastic noise. Their scheme is pithy as the number of species and reactions scale linearly and quadratically respectively, with the number of states in the FSA. They also present a variant of their scheme where the input species are not consumed, and instead catalytically trigger a cascade of transitions in a buffer system, which in turn mediates the actual transitions in the FSA. The problem is an important one to solve, as FSA are basic building blocks of computation. Understanding how to implement FSA by reaction networks allows bringing the computational lens to an examination of chemical computing, including processes in living cells.
The proposed scheme is (unsurprisingly) sensitive to the choice of reaction rates, which the authors empirically characterize through simulations. They have considered the various ways their compiled CRN could fail and have then tuned the reaction rates to avoid these. This method of setting the rates wouldn't scale well with the complexity of the FSA. However the authors do mention that picking rates K_AM = 1, K_Flip = 10 and K_input = 100 scaled well in their experience. It would be nicer if they mathematically claimed and proved this.
Finally, they demonstrate their construction by compiling a FSA for accepting the square root of a 4-bit number into a CRN via their scheme, which are the kind of things that are always a delight to see.
The construction is simple yet very interesting. It is also pithy as the number of species only scales linearly and the number of reactions only scales quadratically.
To summarize, this is a simple but important advance. I imagine lots of papers will build on this work in future, and strongly recommend publication.

Decision letter (RSOS-211310.R0)
We hope you are keeping well at this difficult and unusual time. We continue to value your support of the journal in these challenging circumstances. If Royal Society Open Science can assist you at all, please don't hesitate to let us know at the email address below.

Dear Dr Lakin
On behalf of the Editors, we are pleased to inform you that your Manuscript RSOS-211310 "Robust finite automata in stochastic chemical reaction networks" has been accepted for publication in Royal Society Open Science subject to minor revision in accordance with the referees' reports. Please find the referees' comments along with any feedback from the Editors below my signature.
We invite you to respond to the comments and revise your manuscript. Below the referees' and Editors' comments (where applicable) we provide additional requirements. Final acceptance of your manuscript is dependent on these requirements being met. We provide guidance below to help you prepare your revision.
Please submit your revised manuscript and required files (see below) no later than 7 days from today's (ie 10-Nov-2021) date. Note: the ScholarOne system will 'lock' if submission of the revision is attempted 7 or more days after the deadline. If you do not think you will be able to meet this deadline please contact the editorial office immediately.
Please note article processing charges apply to papers accepted for publication in Royal Society Open Science (https://royalsocietypublishing.org/rsos/charges). Charges will also apply to papers transferred to the journal from other Royal Society Publishing journals, as well as papers submitted as part of our collaboration with the Royal Society of Chemistry (https://royalsocietypublishing.org/rsos/chemistry). Fee waivers are available but must be requested when you submit your revision (https://royalsocietypublishing.org/rsos/waivers). The reviewers offer a few recommendations to improve the paper. Please consider them carefully and respond to their suggestions point by point.
Reviewer comments to Author: Reviewer: 1 Comments to the Author(s) In the paper, the authors demonstrated the principles and modules needed for building finite state machines with chemical reaction networks. The paper is nicely written and is advised to be published with following suggestions: The authors first introduced the major logic that enables the implementation of finite state machines -the approximate majority algorithm, where the purpose of such an algorithm is to allow the system to converge to a single state. While the principle works flawlessly in-silico, one minor comment here, as this is a computational paper, is the difficulty in actual implementation with chemical or biochemical molecules. One may need to think carefully about how to design reversible molecules that would react following the proposed rules. It would thus help if the authors may provide some thoughts on the implementation if possible.
The second key module the authors implemented for building finite state machines is the transition mapping and introduction of "flip species". The authors demonstrated how the introduction of new species stabilizes the state transitions of finite state machines by showing the two cases of simulation with and without the implementation of "flip species" -with the noninclusion case erring with failed state transition. While this is a straightforward example, the stochasticity of the system may render this result difficult to reproduce. It would thus greatly help readers to visualize the noise and show the stabilization effect for running the two designs with multiple runs, then use the derived mean and standard deviation to visualize stability effect of flip species introduction provides, thus, a major comment here would be the introduction of such a figure added in figure 3. Also, it is not clear how the simulations of the non-flip-speciescases were conducted, this is essential to include for readers' reference.
In the following sections, the authors performed detailed parametric studies and characterization to learn how different rates affect the state-switching robustness of the finite state machines. The heatmaps offer clear visualization to learn the trend and the authors also provide thoughtful interpretations. However, another major comment here is that how the authors are evaluating the FSM correctness is unclear, and should be carefully explained. It is clear to interpret 100% and 0% correctness, but there are multiple regions with values lying in-between and is confusing for readers -for example, did the FSMs skip some states or did they fail to skip states? The authors should clarify the definition in the main text or in the captions below.
Finally, the implementation of solving the square root problem is a great and interesting example demonstrating the power of useful finite state machines with CRN implementations! Reviewer: 2 Comments to the Author(s) The authors present a scheme to compile arbitrary Finite State Automata (FSA) into stochastic Chemical Reaction networks (CRN). They do this by using Approximate Majority to stabilize the state of the system after partial transitions are triggered by an Input. This makes the construction robust to stochastic noise. Their scheme is pithy as the number of species and reactions scale linearly and quadratically respectively, with the number of states in the FSA. They also present a variant of their scheme where the input species are not consumed, and instead catalytically trigger a cascade of transitions in a buffer system, which in turn mediates the actual transitions in the FSA. The problem is an important one to solve, as FSA are basic building blocks of computation. Understanding how to implement FSA by reaction networks allows bringing the computational lens to an examination of chemical computing, including processes in living cells.
The proposed scheme is (unsurprisingly) sensitive to the choice of reaction rates, which the authors empirically characterize through simulations. They have considered the various ways their compiled CRN could fail and have then tuned the reaction rates to avoid these. This method of setting the rates wouldn't scale well with the complexity of the FSA. However the authors do mention that picking rates K_AM = 1, K_Flip = 10 and K_input = 100 scaled well in their experience. It would be nicer if they mathematically claimed and proved this.
Finally, they demonstrate their construction by compiling a FSA for accepting the square root of a 4-bit number into a CRN via their scheme, which are the kind of things that are always a delight to see.
The construction is simple yet very interesting. It is also pithy as the number of species only scales linearly and the number of reactions only scales quadratically.
To summarize, this is a simple but important advance. I imagine lots of papers will build on this work in future, and strongly recommend publication.

===PREPARING YOUR MANUSCRIPT===
Your revised paper should include the changes requested by the referees and Editors of your manuscript.
You should provide two versions of this manuscript and both versions must be provided in an editable format: one version should clearly identify all the changes that have been made (for instance, in coloured highlight, in bold text, or tracked changes); a 'clean' version of the new manuscript that incorporates the changes made, but does not highlight them. This version will be used for typesetting.
Please ensure that any equations included in the paper are editable text and not embedded images.
Please ensure that you include an acknowledgements' section before your reference list/bibliography. This should acknowledge anyone who assisted with your work, but does not qualify as an author per the guidelines at https://royalsociety.org/journals/ethicspolicies/openness/.
While not essential, it will speed up the preparation of your manuscript proof if you format your references/bibliography in Vancouver style (please see https://royalsociety.org/journals/authors/author-guidelines/#formatting). You should include DOIs for as many of the references as possible.
If you have been asked to revise the written English in your submission as a condition of publication, you must do so, and you are expected to provide evidence that you have received language editing support. The journal would prefer that you use a professional language editing service and provide a certificate of editing, but a signed letter from a colleague who is a proficient user of English is acceptable. Note the journal has arranged a number of discounts for authors using professional language editing services (https://royalsociety.org/journals/authors/benefits/language-editing/).

===PREPARING YOUR REVISION IN SCHOLARONE===
To revise your manuscript, log into https://mc.manuscriptcentral.com/rsos and enter your Author Centre -this may be accessed by clicking on "Author" in the dark toolbar at the top of the page (just below the journal name). You will find your manuscript listed under "Manuscripts with Decisions". Under "Actions", click on "Create a Revision".
Attach your point-by-point response to referees and Editors at the 'View and respond to decision letter' step. This document should be uploaded in an editable file type (.doc or .docx are preferred). This is essential, and your manuscript will be returned to you if you do not provide it.
Please ensure that you include a summary of your paper at the 'Type, Title, & Abstract' step. This should be no more than 100 words to explain to a non-scientific audience the key findings of your research. This will be included in a weekly highlights email circulated by the Royal Society press office to national UK, international, and scientific news outlets to promote your work. An effective summary can substantially increase the readership of your paper.
At the 'File upload' step you should include the following files: --Your revised manuscript in editable file format (.doc, .docx, or .tex preferred). You should upload two versions: 1) One version identifying all the changes that have been made (for instance, in coloured highlight, in bold text, or tracked changes); 2) A 'clean' version of the new manuscript that incorporates the changes made, but does not highlight them.
--An individual file of each figure (EPS or print-quality PDF preferred [either format should be produced directly from original creation package], or original software format).
--An editable file of all figure and table captions. Note: you may upload the figure, table, and caption files in a single Zip folder.
--If you are requesting a discretionary waiver for the article processing charge, the waiver form must be included at this step.
--If you are providing image files for potential cover images, please upload these at this step, and inform the editorial office you have done so. You must hold the copyright to any image provided.
--A copy of your point-by-point response to referees and Editors. This will expedite the preparation of your proof.
At the 'Details & comments' step, you should review and respond to the queries on the electronic submission form. In particular, we would ask that you do the following: --Ensure that your data access statement meets the requirements at https://royalsociety.org/journals/authors/author-guidelines/#data. You should ensure that you cite the dataset in your reference list. If you have deposited data etc in the Dryad repository, please only include the 'For publication' link at this stage. You should remove the 'For review' link.
--If you are requesting an article processing charge waiver, you must select the relevant waiver option (if requesting a discretionary waiver, the form should have been uploaded, see 'File upload' above).
--If you have uploaded any electronic supplementary (ESM) files, please ensure you follow the guidance at https://royalsociety.org/journals/authors/author-guidelines/#supplementarymaterial to include a suitable title and informative caption. An example of appropriate titling and captioning may be found at https://figshare.com/articles/Table_S2_from_Is_there_a_trade-off_between_peak_performance_and_performance_breadth_across_temperatures_for_aerobic_sc ope_in_teleost_fishes_/3843624. At the 'Review & submit' step, you must view the PDF proof of the manuscript before you will be able to submit the revision. Note: if any parts of the electronic submission form have not been completed, these will be noted by red message boxes -you will need to resolve these errors before you can submit the revision.

Decision letter (RSOS-211310.R1)
We hope you are keeping well at this difficult and unusual time. We continue to value your support of the journal in these challenging circumstances. If Royal Society Open Science can assist you at all, please don't hesitate to let us know at the email address below.
Dear Dr Lakin, I am pleased to inform you that your manuscript entitled "Robust finite automata in stochastic chemical reaction networks" is now accepted for publication in Royal Society Open Science.
Please remember to make any data sets or code libraries 'live' prior to publication, and update any links as needed when you receive a proof to check -for instance, from a private 'for review' URL to a publicly accessible 'for publication' URL. It is good practice to also add data sets, code and other digital materials to your reference list.
Our payments team will be in touch shortly if you are required to pay a fee for the publication of the paper (if you have any queries regarding fees, please see https://royalsocietypublishing.org/rsos/charges or contact authorfees@royalsociety.org).
The proof of your paper will be available for review using the Royal Society online proofing system and you will receive details of how to access this in the near future from our production office (openscience_proofs@royalsociety.org). We aim to maintain rapid times to publication after acceptance of your manuscript and we would ask you to please contact both the production office and editorial office if you are likely to be away from e-mail contact to minimise delays to publication. If you are going to be away, please nominate a co-author (if available) to manage the proofing process, and ensure they are copied into your email to the journal.
Please see the Royal Society Publishing guidance on how you may share your accepted author manuscript at https://royalsociety.org/journals/ethics-policies/media-embargo/. After publication, some additional ways to effectively promote your article can also be found here https://royalsociety.org/blog/2020/07/promoting-your-latest-paper-and-tracking-yourresults/.
On behalf of the Editors of Royal Society Open Science, thank you for your support of the journal and we look forward to your continued contributions to Royal Society Open Science. This letter supports the resubmission of our revised manuscript entitled "Robust finite automata in stochastic chemical reaction networks" to Royal Society Open Science.
We thank the reviewers for their positive response to the initial version of our manuscript. In preparing this revision, we have addressed the points raised in their reviews, as outlined below.
Reviewer: 1 Comments to the Author(s) In the paper, the authors demonstrated the principles and modules needed for building finite state machines with chemical reaction networks. The paper is nicely written and is advised to be published with following suggestions: The authors first introduced the major logic that enables the implementation of finite state machines -the approximate majority algorithm, where the purpose of such an algorithm is to allow the system to converge to a single state. While the principle works flawlessly in-silico, one minor comment here, as this is a computational paper, is the difficulty in actual implementation with chemical or biochemical molecules. One may need to think carefully about how to design reversible molecules that would react following the proposed rules. It would thus help if the authors may provide some thoughts on the implementation if possible.
We thank the reviewer for their positive recommendation of our manuscript. We have addressed this question of implementability in the Discussion (subsection entititled "Possibility of wet chemistry implementations") by adding the following text on how our design could be converted into a DNA strand displacement-based implementation: "That paper shows that arbitrary abstract chemical reaction networks can be directly implemented using strand displacement networks, thereby providing a direct route from our design to a laboratory implementation, at least in principle. This approach would require careful sequence design to ensure that the system adheres to the reaction rules specified here." The second key module the authors implemented for building finite state machines is the transition mapping and introduction of "flip species". The authors demonstrated how the introduction of new species stabilizes the state transitions of finite state machines by showing the two cases of simulation with and without the implementation of âȂIJflip speciesâȂİ -with the non-inclusion case erring with failed state transition. While this is a straightforward example, the stochasticity of the system may render this result difficult to reproduce. It would thus greatly help readers to visualize the noise and show the stabilization effect for running the two designs with multiple runs, then use the derived mean and standard deviation to visualize stability effect of flip species introduction provides, thus, a major comment here would be the introduction of such a figure added in figure 3. This is an excellent suggestion and we have included precisely these plots as Figure 3c and 3d in the revised version of the manuscript. As expected, these show tight error bars around the correct sequence of state transitions when the flip species are used and broad error bars and indeterminate average states (due to incorrect transitions into different states in each trace) in the averaged system when the flip species are not used. This addition demonstrates more convincing statistical evidence for the utility of the flip species approach and we thank the review for suggesting it.
Also, it is not clear how the simulations of the non-flip-species-cases were conducted, this is essential to include for readers' reference.
This omission has been corrected by clarifying the explanation of the CRN that was used for the "no flip" simulations, by adding the following text: "In other words, in the "no flip" case, the transition mapping and transition modules from Figure 2 are replaced with the following three reactions", followed by a listing of the corresponding three reactions.
In the following sections, the authors performed detailed parametric studies and characterization to learn how different rates affect the state-switching robustness of the finite state machines. The heatmaps offer clear visualization to learn the trend and the authors also provide thoughtful interpretations. However, another major comment here is that how the authors are evaluating the FSM correctness is unclear, and should be carefully explained. It is clear to interpret 100% and 0% correctness, but there are multiple regions with values lying in-between and is confusing for readersfor example, did the FSMs skip some states or did they fail to skip states? The authors should clarify the definition in the main text or in the captions below.
We agree that we did not explain clearly how correct state transitions are detected in the simulation. To address this, we added the following text to the relevant part of the manuscript: "Correct state transitions were detected by averaging the species counts from the middle fifty percent of time points occurring between two perturbations. For a given time point, if the count of the relevant state species is greater than 80% of the total number of state species (which in this case means 80 copies), then we say the system is in the corresponding state." Finally, the implementation of solving the square root problem is a great and interesting example demonstrating the power of useful finite state machines with CRN implementations!
We are glad that the reviewer appreciated this nod to molecular computing history.
Reviewer: 2 Comments to the Author(s) The authors present a scheme to compile arbitrary Finite State Automata (FSA) into stochastic Chemical Reaction networks (CRN). They do this by using Approximate Majority to stabilize the state of the system after partial transitions are triggered by an Input. This makes the construction robust to stochastic noise. Their scheme is pithy as the number of species and reactions scale linearly and quadratically respectively, with the number of states in the FSA. They also present a variant of their scheme where the input species are not consumed, and instead catalytically trigger a cascade of transitions in a buffer system, which in turn mediates the actual transitions in the FSA. The problem is an important one to solve, as FSA are basic building blocks of computation. Understanding how to implement FSA by reaction networks allows bringing the computational lens to an examination of chemical computing, including processes in living cells.
We appreciate the positive comments on the scalability of the system design in terms of the numbers of spceies and reactions.
The proposed scheme is (unsurprisingly) sensitive to the choice of reaction rates, which the authors empirically characterize through simulations. They have considered the various ways their compiled CRN could fail and have then tuned the reaction rates to avoid these. This method of setting the rates wouldn't scale well with the complexity of the FSA. However the authors do mention that picking rates K_AM = 1, K_Flip = 10 and K_input = 100 scaled well in their experience. It would be nicer if they mathematically claimed and proved this.
While it is true that the performance of our stochastic implementations of finite state automata are dependent on the rate constant values, we note that the question of scaling with complexity is not a significant barrier to the application of our technique since the number of rates required does not increase with increasing size of the implemented FSA. Moreover, although we do observe slightly differnet "performance heatmaps" for the n-state cycle FSA examples, there is a reasonable degree of overlap which means that rate constants are transferrable between different FSA designs to a significant extent. For example, the rate constants used in the square root FSA implementation example were not optimized for that specific example but rather transferred from an earlier system. As a general rule of thumb, increasing the separation of timescales between rates tends to improve accuracy of the FSA state transitions. With regard to mathematical proof, we note that even for 100 copies of each molecule, the state space for the system will be far too large to admit analytical solutions for system accuracy.
Finally, they demonstrate their construction by compiling a FSA for accepting the square root of a 4-bit number into a CRN via their scheme, which are the kind of things that are always a delight to see. The construction is simple yet very interesting. It is also pithy as the number of species only scales linearly and the number of reactions only scales quadratically. To summarize, this is a simple but important advance. I imagine lots of papers will build on this work in future, and strongly recommend publication.