Memory-two zero-determinant strategies in repeated games

Repeated games have provided an explanation of how mutual cooperation can be achieved even if defection is more favourable in a one-shot game in the Prisoner’s Dilemma situation. Recently found zero-determinant (ZD) strategies have substantially been investigated in evolutionary game theory. The original memory-one ZD strategies unilaterally enforce linear relationships between average pay-offs of players. Here, we extend the concept of ZD strategies to memory-two strategies in repeated games. Memory-two ZD strategies unilaterally enforce linear relationships between correlation functions of pay-offs and pay-offs of the previous round. Examples of memory-two ZD strategy in the repeated Prisoner’s Dilemma game are provided, some of which generalize the tit-for-tat strategy to a memory-two case. Extension of ZD strategies to memory-n case with n ≥ ~2 is also straightforward.

1. Line #32~33: ``when the opponent `unilaterally' defects twice.'' 2. If we think of the conventional prisoner's dilemma game, Eq. (3.6) seems over-determined because it contains 16 equations with 11 unknowns. Is it possible to specify conditions for the solution to exist?
3. Although Eq. (3.6) clearly shows an algebraic structure of the memory-two ZD strategies, I wonder if we really need the product of $s_b$ and $s_c$ on the right-hand side. There may be a degree of freedom to choose another function of $s_b$ and $s_c$.

Recommendation?
Major revision is needed (please make suggestions in comments)

Comments to the Author(s)
I would like to support this work because it contains solid calculation, but I strongly believe that present version is not appropriate for a general reader of R. Soc. Open Sci. Please note that this journal has highly interdisciplinary character with broad range of readers from rather different backgrounds. Therefore it is essential to present works in a way that is reachable for an ordinary reader.
For example the intro seems to be too brief and it is hard to find out what was really known earlier and what was the proper inspiration of present calculations. For example several previous works were ignored which also discussed extortion or TFT strategy in the mentioned social dilemma. Without giving a full list, let me just mentione some representative papers: Sci. Rep. 4 (2014)  It would be important to stress that present calculation assumes well-mixed populations while results may change in structured populations where interactions are fairly fixed and limited (see J. R. Soc. Interface 10 (2013) 20120997 and EPL 131 (2020) 68001).
In general I found that the figures are very poor and not meaningful. They just simply illustrate that the numerical iterations reached a saturation value very early. But yields nothing more. I think it would be more insightful to present the payoff values in dependence of the model parameter instead of presenting a straightforward constant values in time.
The discussion part does not serve its original goal either. Actually it is not a proper discussion. Technical details should go to an appendix. Instead, it would be wiser to summarize the main findings here and compare them with earlier works. For instance the application of memory has a huge history in evolutionary game theory and related research already produced several interesting works, like Physica A 389 (2010) 2390-2396; Sci. Rep. 9 (2019) 262; Chaos, Solitons and Fractals 130 (2020) 109447. A related comment and a brief discussion about the general role of memory would be really useful. I strongly believe that by addressing the critical points I raised will result in a significantly improved presentation which would expect a better response from the scientific community. Because the present version is very far the level that is expected from a paper to be published in this nice journal.

Decision letter (RSOS-202186.R0)
We hope you are keeping well at this difficult and unusual time. We continue to value your support of the journal in these challenging circumstances. If Royal Society Open Science can assist you at all, please don't hesitate to let us know at the email address below.

Dear Dr Ueda
The Editors assigned to your paper RSOS-202186 "Memory-two zero-determinant strategies in repeated games" have now received comments from reviewers and would like you to revise the paper in accordance with the reviewer comments and any comments from the Editors. Please note this decision does not guarantee eventual acceptance.
We invite you to respond to the comments supplied below and revise your manuscript. Below the referees' and Editors' comments (where applicable) we provide additional requirements. Final acceptance of your manuscript is dependent on these requirements being met. We provide guidance below to help you prepare your revision.
We do not generally allow multiple rounds of revision so we urge you to make every effort to fully address all of the comments at this stage. If deemed necessary by the Editors, your manuscript will be sent back to one or more of the original reviewers for assessment. If the original reviewers are not available, we may invite new reviewers.
Please submit your revised manuscript and required files (see below) no later than 21 days from today's (ie 02-Mar-2021) date. Note: the ScholarOne system will 'lock' if submission of the revision is attempted 21 or more days after the deadline. If you do not think you will be able to meet this deadline please contact the editorial office immediately.
Please note article processing charges apply to papers accepted for publication in Royal Society Open Science (https://royalsocietypublishing.org/rsos/charges). Charges will also apply to papers transferred to the journal from other Royal Society Publishing journals, as well as papers submitted as part of our collaboration with the Royal Society of Chemistry (https://royalsocietypublishing.org/rsos/chemistry). Fee waivers are available but must be requested when you submit your revision (https://royalsocietypublishing.org/rsos/waivers).
Thank you for submitting your manuscript to Royal Society Open Science and we look forward to receiving your revision. If you have any questions at all, please do not hesitate to get in touch. Comments to the Author(s) The author has investigated memory-$n$ zero-determinant (ZD) strategies with general $n>1$. The finding is that such strategies enforce linear relations between correlation functions of payoffs over $n$ time steps. Overall, the manuscript is clearly written about an interesting topic, and such understanding has great potential importance if we consider the recent development of memory-$n$ strategies that overcome shortcomings of the original ZD strategies. I did not follow all the details of the calculation, but the result looks convincing enough according to the numerical calculation. I recommend publication of this manuscript. Here are some minor comments that I would like the author to consider: 1. Line #32~33: ``when the opponent `unilaterally' defects twice.'' 2. If we think of the conventional prisoner's dilemma game, Eq. (3.6) seems over-determined because it contains 16 equations with 11 unknowns. Is it possible to specify conditions for the solution to exist?
3. Although Eq. (3.6) clearly shows an algebraic structure of the memory-two ZD strategies, I wonder if we really need the product of $s_b$ and $s_c$ on the right-hand side. There may be a degree of freedom to choose another function of $s_b$ and $s_c$.
Reviewer: 2 Comments to the Author(s) I would like to support this work because it contains solid calculation, but I strongly believe that present version is not appropriate for a general reader of R. Soc. Open Sci. Please note that this journal has highly interdisciplinary character with broad range of readers from rather different backgrounds. Therefore it is essential to present works in a way that is reachable for an ordinary reader.
For example the intro seems to be too brief and it is hard to find out what was really known earlier and what was the proper inspiration of present calculations. For example several previous works were ignored which also discussed extortion or TFT strategy in the mentioned social dilemma. Without giving a full list, let me just mentione some representative papers: Sci. Rep. 4 (2014) 5496; Nat. Commun. 10 (2019)  In general I found that the figures are very poor and not meaningful. They just simply illustrate that the numerical iterations reached a saturation value very early. But yields nothing more. I think it would be more insightful to present the payoff values in dependence of the model parameter instead of presenting a straightforward constant values in time.
The discussion part does not serve its original goal either. Actually it is not a proper discussion. Technical details should go to an appendix. Instead, it would be wiser to summarize the main findings here and compare them with earlier works. For instance the application of memory has a huge history in evolutionary game theory and related research already produced several interesting works, like Physica A 389 (2010)  I strongly believe that by addressing the critical points I raised will result in a significantly improved presentation which would expect a better response from the scientific community. Because the present version is very far the level that is expected from a paper to be published in this nice journal.

===PREPARING YOUR MANUSCRIPT===
Your revised paper should include the changes requested by the referees and Editors of your manuscript. You should provide two versions of this manuscript and both versions must be provided in an editable format: one version identifying all the changes that have been made (for instance, in coloured highlight, in bold text, or tracked changes); a 'clean' version of the new manuscript that incorporates the changes made, but does not highlight them. This version will be used for typesetting if your manuscript is accepted.
Please ensure that any equations included in the paper are editable text and not embedded images.
Please ensure that you include an acknowledgements' section before your reference list/bibliography. This should acknowledge anyone who assisted with your work, but does not qualify as an author per the guidelines at https://royalsociety.org/journals/ethicspolicies/openness/.
While not essential, it will speed up the preparation of your manuscript proof if accepted if you format your references/bibliography in Vancouver style (please see https://royalsociety.org/journals/authors/author-guidelines/#formatting). You should include DOIs for as many of the references as possible.
If you have been asked to revise the written English in your submission as a condition of publication, you must do so, and you are expected to provide evidence that you have received language editing support. The journal would prefer that you use a professional language editing service and provide a certificate of editing, but a signed letter from a colleague who is a native speaker of English is acceptable. Note the journal has arranged a number of discounts for authors using professional language editing services (https://royalsociety.org/journals/authors/benefits/language-editing/).

===PREPARING YOUR REVISION IN SCHOLARONE===
To revise your manuscript, log into https://mc.manuscriptcentral.com/rsos and enter your Author Centre -this may be accessed by clicking on "Author" in the dark toolbar at the top of the page (just below the journal name). You will find your manuscript listed under "Manuscripts with Decisions". Under "Actions", click on "Create a Revision".
Attach your point-by-point response to referees and Editors at Step 1 'View and respond to decision letter'. This document should be uploaded in an editable file type (.doc or .docx are preferred). This is essential.
Please ensure that you include a summary of your paper at Step 2 'Type, Title, & Abstract'. This should be no more than 100 words to explain to a non-scientific audience the key findings of your research. This will be included in a weekly highlights email circulated by the Royal Society press office to national UK, international, and scientific news outlets to promote your work.

At
Step 3 'File upload' you should include the following files: --Your revised manuscript in editable file format (.doc, .docx, or .tex preferred). You should upload two versions: 1) One version identifying all the changes that have been made (for instance, in coloured highlight, in bold text, or tracked changes); 2) A 'clean' version of the new manuscript that incorporates the changes made, but does not highlight them.
--An individual file of each figure (EPS or print-quality PDF preferred [either format should be produced directly from original creation package], or original software format).
--An editable file of each table (.doc, .docx, .xls, .xlsx, or .csv --If you are requesting a discretionary waiver for the article processing charge, the waiver form must be included at this step.
--If you are providing image files for potential cover images, please upload these at this step, and inform the editorial office you have done so. You must hold the copyright to any image provided.
--A copy of your point-by-point response to referees and Editors. This will expedite the preparation of your proof.

At
Step 6 'Details & comments', you should review and respond to the queries on the electronic submission form. In particular, we would ask that you do the following: --Ensure that your data access statement meets the requirements at https://royalsociety.org/journals/authors/author-guidelines/#data. You should ensure that you cite the dataset in your reference list. If you have deposited data etc in the Dryad repository, please include both the 'For publication' link and 'For review' link at this stage.
--If you are requesting an article processing charge waiver, you must select the relevant waiver option (if requesting a discretionary waiver, the form should have been uploaded at Step 3 'File upload' above).
--If you have uploaded ESM files, please ensure you follow the guidance at https://royalsociety.org/journals/authors/author-guidelines/#supplementary-material to include a suitable title and informative caption. An example of appropriate titling and captioning may be found at https://figshare.com/articles/Table_S2_from_Is_there_a_trade-off_between_peak_performance_and_performance_breadth_across_temperatures_for_aerobic_sc ope_in_teleost_fishes_/3843624.

At
Step 7 'Review & submit', you must view the PDF proof of the manuscript before you will be able to submit the revision. Note: if any parts of the electronic submission form have not been completed, these will be noted by red message boxes.

Do you have any ethical concerns with this paper? No
Have you any concerns about statistical analyses in this paper? No

Recommendation?
Accept as is

Comments to the Author(s)
It is a significantly improved version. The author has addressed the critical points successfully, hence I am happy to support publication of the revised version.

Decision letter (RSOS-202186.R1)
We hope you are keeping well at this difficult and unusual time. We continue to value your support of the journal in these challenging circumstances. If Royal Society Open Science can assist you at all, please don't hesitate to let us know at the email address below.
Dear Dr Ueda, It is a pleasure to accept your manuscript entitled "Memory-two zero-determinant strategies in repeated games" in its current form for publication in Royal Society Open Science. The comments of the reviewer(s) who reviewed your manuscript are included at the foot of this letter.
Please ensure that you send to the editorial office an editable version of your accepted manuscript, and individual files for each figure and table included in your manuscript. You can send these in a zip folder if more convenient. Failure to provide these files may delay the processing of your proof. You may disregard this request if you have already provided these files to the editorial office.
You can expect to receive a proof of your article in the near future. Please contact the editorial office (openscience@royalsociety.org) and the production office (openscience_proofs@royalsociety.org) to let us know if you are likely to be away from e-mail contact --if you are going to be away, please nominate a co-author (if available) to manage the proofing process, and ensure they are copied into your email to the journal.
Due to rapid publication and an extremely tight schedule, if comments are not received, your paper may experience a delay in publication.
Please see the Royal Society Publishing guidance on how you may share your accepted author manuscript at https://royalsociety.org/journals/ethics-policies/media-embargo/. After publication, some additional ways to effectively promote your article can also be found here https://royalsociety.org/blog/2020/07/promoting-your-latest-paper-and-tracking-yourresults/. In general I found that the figures are very poor and not meaningful. They just simply illustrate that the numerical iterations reached a saturation value very early. But yields nothing more. I think it would be more insightful to present the payoff values in dependence of the model parameter instead of presenting a straightforward constant values in time. [Our reply:] In our revised version, we have introduced a parameter q to the strategy of player 2 in Eq. (4.5) and changed q in the range [0, 1] in our numerical simulations. Accordingly the horizontal axes of Figures 1-5 have been changed to q. We can check that linear relations hold for all q.
[Referee's comment:] The discussion part does not serve its original goal either. Actually it is not a proper discussion. Technical details should go to an appendix. Instead, it would be wiser to summarize the main findings here and compare them with earlier works. For instance the application of memory has a huge history in evolutionary game theory and related research already produced several interesting works, like Physica A 389 (2010) 2390-2396; Sci. Rep. 9 (2019) 262; Chaos, Solitons and Fractals 130 (2020) 109447. A related comment and a brief discussion about the general role of memory would be really useful. [Our reply:] In our revised version, we have summarized our main findings and discussed roles of memory in Section 6. Technical details about memory-n ZD strategies have been moved to Section 5.