Learned valuation during forage decision-making in cuttlefish

Decision-making, when humans and other animals choose between two options, is not always based on the absolute values of the options but can also depend on their relative values. The present study examines whether decision-making by cuttlefish is dependent on relative values learned from previous experience. Cuttlefish preferred a larger quantity when making a choice between one or two shrimps (1 versus 2) during a two-alternative forced choice. However, after cuttlefish were primed under conditions where they were given a small reward for choosing one shrimp in a no shrimp versus one shrimp test (0 versus 1) six times in a row, they chose one shrimp significantly more frequently in the 1 versus 2 test. This reversed preference for a smaller quantity was not due to satiation at the time of decision-making, as cuttlefish fed a small shrimp six times without any choice test prior to the experiment still preferred two shrimps significantly more often in a subsequent 1 versus 2 test. This suggests that the preference of one shrimp in the quantity comparison test occurs via a process of learned valuation. Foraging preference in cuttlefish thus depends on the relative value of previous prey choices.

Lines 259-261: please consider revising the sentence. Currently, it's a little confusing. What exactly is helping the cuttlefish optimise their foraging strategy? Reference-dependent evaluation or value-based decision making or both? Lines 267-270: please consider revising the last sentence. The statement requires expanding and further justification. Why is foraging cognition in cuttlefish equivalent to that of vertebrates? What specific aspects are equivalent? Are the authors referring to the previous statement in that state-dependent valuation learning is rife across diverse taxa? If yes, then foraging cognition in cuttlefish is not only akin to vertebrates but also other invertebrates i.e. insects. If no, what specific aspects of learned valuation is vertebrate-like? Why is this important? Lines 365-373: Please revise the figure 2 legend to clearly state that cuttlefish in the experiment depicted in 2D did not get exposed to a priming effect.
Decision letter (RSOS-200917.R0) We hope you are keeping well at this difficult and unusual time. We continue to value your support of the journal in these challenging circumstances. If Royal Society Open Science can assist you at all, please don't hesitate to let us know at the email address below.

Dear Dr Chiao:
Manuscript ID RSOS-200917 entitled "Learned valuation during forage decision making in cuttlefish" which you submitted to Royal Society Open Science, has been reviewed. The comments from reviewers are included at the bottom of this letter.
In view of the criticisms of the reviewers, the manuscript has been rejected in its current form. However, a new manuscript may be submitted which takes into consideration these comments.
Please note that resubmitting your manuscript does not guarantee eventual acceptance, and that your resubmission will be subject to peer review before a decision is made.
You will be unable to make your revisions on the originally submitted version of your manuscript. Instead, revise your manuscript and upload the files via your author centre.
Once you have revised your manuscript, go to https://mc.manuscriptcentral.com/rsos and login to your Author Center. Click on "Manuscripts with Decisions," and then click on "Create a Resubmission" located next to the manuscript number. Then, follow the steps for resubmitting your manuscript.
Your resubmitted manuscript should be submitted by 18-Jan-2021. If you are unable to submit by this date please contact the Editorial Office.
We look forward to receiving your resubmission. Comments to the Author: I have now received two reviews of your manuscript entitled "Learned valuation during forage decision making in cuttlefish". Referee #1 expressed concerns about the appropriateness of invoking "less is more" and "loss aversion" arguments when interpreting the results. Upon careful reading of the manuscript, I felt that the description of the methods was not sufficiently clear to allow me to assess this. Referee #2 also pointed out that several aspects of the methods are unclear. Some of the details that are unclear after reading the manuscript make it impossible to assess the soundness of the interpretations that have been drawn from the results. As such, I am recommending that the manuscript be rejected with possibility to resubmit following substantial clarification of the methods.
Should you choose to resubmit the manuscript, please address the following points, in addition to the very detailed points provided by referee #2. 1. A schematic representation of the study setup would be useful -including the treatment orders, choices provided, rewards associated with each choice, and sample sizes (number of fish) and number of trials. 2. Please be explicit about what total rewards were offered in each phase, including, for example, whether a single shrimp choice plus the additional reward of a "tiny" shrimp was equivalent to, or less than the reward experienced when choosing 2 shrimp in the baseline preference test (or were these trials all unrewarded?). This is critical for interpreting the change in preference in the third phase (preference for 1 shrimp over 2 shrimp) as the interpretation if 2 shrimp was never rewarded is very different than if 2 shrimp was rewarded with 2 shrimp in the baseline trial. If the former, then the study demonstrates associative learning, if the latter, then knowing how much the "tiny" shrimp was worth becomes critical.
Reviewers' Comments to Author: Reviewer: 1 Comments to the Author(s) This manuscript simply shows that cuttlefish are sensitive to positive reinforcement. They repeated the actions reinforced during "priming" in the critical 1 vs. 2 tests. The authors acknowledge this in the discussion. Yet, they decided to frame the entire manuscript in cognitive and ecological terms. Given that reinforcement learning can account for the entire set of findings appealing to behavioural economics, less is more, loss aversion and the like seems inappropriate. In the discussion, the authors write "this learned valuation effect was likely to dependent on the animals' cognitive processes…". My reading is that the effect was most likely caused by prior positive reinforcement of the actions displayed at test. Morgan's Canon may help us decide.

Reviewer: 2
Comments to the Author(s) RS0S-200917: Learned valuation during forage decision making in cuttlefish This is a comprehensive and important study. The authors use an alternative force two-choice paradigm to investigate learned valuation in cuttlefish. The experiment is well-designed and the evidence for the conclusions is convincing. My main concerns outlined below can hopefully be addressed with some additional explanation in the text. The writing requires some attention as some sections are poorly worded. I have made some suggestions and provide comments that are aimed to help the authors improve the manuscript. Specific comments Line 49: consider revising the end of this sentence to 'this approach can maximize fitness'.
Lines 50-52: consider revising these two sentences to form a single succinct sentence: 'However, animals might also optimize their foraging strategy via more complex mechanisms including spatial memory, value-based decision-making, and executive control when exploiting food resources.' Line 71: Please remove the word 'constantly' as this suggests that cuttlefish spend most of their time foraging. Accelerometry research has shown that cuttlefish have long rest periods that are punctuated by brief foraging bouts (Aitken, O'Dor & Jackkson, 2005). Line 79: insert the word 'it' i.e. 'by coating it with quinine'. Lines 79-84: this sentence is poorly worded, please revise and make it clear that you are referring to a different study than the previous quinine study. i.e. 'Moreover, when cuttlefish are faced with a trade-off between one large shrimp and two small shrimp, they choose the single larger shrimp when they are hungry but choose the two smaller shrimp when they are satiated.' Line 82: omit the word 'previous'. Line 85: best to refer to it as 'what', 'where' and 'when' components as episodic-like memory can often be termed what-where-when memory. Line 86: omit the word 'an' before 'episodic-like memory. Also, omit the word 'very' before 'recent study'. Line 87: omit the words 'are able to' and just say 'cuttlefish adopt flexible foraging…' Line 91: omit the words 'and we did this' Line 93: consider revising to 'Specifically, we manipulated cuttlefish to prefer one shrimp in a choice between one and two shrimp by increasing the food value of one shrimp using a priming effect'. Line 94: a space is missing between 'the' and '1'. Line 105: omit the word 'during' before February 2019, one 'during' is sufficient in this sentence. Line 120: salinity is typically reported as parts per thousand (ppt) not %. Line 123: replace the word 'drew' with 'returned'. Line 128: methods are typically written in past tense, thus please consider changing the word 'consists' to 'consisted'. Line 132: in my experience cuttlefish startle easily, did the cuttlefish exhibit any startle behaviours when the floating container was lowered into the home tank? Had the cuttlefish been habituated to the apparatus and lowering process prior to testing? If yes, please include this in the methods. Lines 137-140: this sentence is a little confusing. Was the entire apparatus lifted out of the water? If yes, please include an explanation as to why the cuttlefish were not rewarded with the food within the chamber once a choice was made? I assume it is because the authors did not want to reinforce the subject's choice in this initial stage of the experiment, but this needs to be stated in the methods. Line 143: The study by Schnell et al. 2016. Lateralization of eye use in cuttlefish, is a more appropriate reference to use here because it demonstrates visual lateralization during prey capture, whereas the Jozet-Alves et al. 2012. study demonstrates visual lateralization when searching for shelter in a two-choice arm maze.
Line 163: what exactly constitutes a tiny shrimp? Can you please include parameters? Lines 182-185: Can the authors please provide a little more detail in the statistical section. Did the authors first conduct normality tests to check parametric assumptions? If yes, please state the type of normality test and the results of that test to justify why non-parametric tests were used. Line 191: consider revising to '...cuttlefish innately preferred a larger quantity of prey'. Line 196: why 'apparently'? Consider omitting this word. Lines 220-221: this sentence is poorly worded, please consider revising. i.e. 'in the wild, the prey that cuttlefish feed on are often distributed heterogeneously and different prey types vary significantly in their nutritional value. Line 223: omit the word 'simply'. Line 227: omit the word 'the' before 'prospect theory'; same comment for line 231. Line 232: consider replacing 'status quo' with 'prevailing conditions'. Lines 232-233: due to poor wording, please consider revising the end of sentence to read: 'for instance, in reference to prevailing conditions or former experiences.' Lines 233-Line 238: please consider revising this sentence to read: '…when cuttlefish did not experience a priming effect, they adopted the...' Line 246: consider revising sentence to read: 'the apparent loss aversion effect was prolonged'. Line 249: omit the word 'to' before 'dependent'. Line 250: omit the word 'with' before 'six tiny shrimp'; also consider revising sentence to read: 'when cuttlefish were fed six tiny shrimp without experiencing the priming effect,…' Line 254: consider replacing the word 'given' with 'prevailing' Lines 255-259: please consider revising this section as the sentence it is poorly worded. For instance, consider breaking the sentence into two parts. Please also consider defining 'instrumental conditioning experiment', as a general biological audience that does not study animal cognition might need further explanation. Line 259: missing the word 'of' after 'regardless' Lines 259-261: please consider revising the sentence. Currently, it's a little confusing. What exactly is helping the cuttlefish optimise their foraging strategy? Reference-dependent evaluation or value-based decision making or both? Lines 267-270: please consider revising the last sentence. The statement requires expanding and further justification. Why is foraging cognition in cuttlefish equivalent to that of vertebrates? What specific aspects are equivalent? Are the authors referring to the previous statement in that state-dependent valuation learning is rife across diverse taxa? If yes, then foraging cognition in cuttlefish is not only akin to vertebrates but also other invertebrates i.e. insects. If no, what specific aspects of learned valuation is vertebrate-like? Why is this important? Lines 365-373: Please revise the figure 2 legend to clearly state that cuttlefish in the experiment depicted in 2D did not get exposed to a priming effect.

Author's Response to Decision Letter for (RSOS-200917.R0)
See Appendix A.

Do you have any ethical concerns with this paper? No
Have you any concerns about statistical analyses in this paper? No

Recommendation?
Accept with minor revision (please list in comments)

Comments to the Author(s)
The manuscript has improved substantially. This new version avoids unnecessary claims of higher cognitive processes. I still fail to understand how the results found can be interpreted using prospect theory (page 19). The fact that decision-making is based on relative rather than on absolute metrics does not help us explain how 1 shrimp is better that 2. (by the way, a phenomenon cannot be akin to a theory, lines 233-236). Positive reinforcement is indeed a good and parsimonious explanation. The pending issue is whether the demonstration of reinforcement learning in cuttlefish is newsworthy. I do not think it is, but the science presented in the paper is solid.

Do you have any ethical concerns with this paper? No
Have you any concerns about statistical analyses in this paper? No

Recommendation?
Accept with minor revision (please list in comments)

Comments to the Author(s)
Comments attached (Appendix B).

Decision letter (RSOS-201602.R0)
We hope you are keeping well at this difficult and unusual time. We continue to value your support of the journal in these challenging circumstances. If Royal Society Open Science can assist you at all, please don't hesitate to let us know at the email address below.
Dear Dr Chiao, On behalf of the Editors, we are pleased to inform you that your Manuscript RSOS-201602 "Learned valuation during forage decision making in cuttlefish" has been accepted for publication in Royal Society Open Science subject to minor revision in accordance with the referees' reports. Please find the referees' comments along with any feedback from the Editors below my signature.
We invite you to respond to the comments and revise your manuscript. Below the referees' and Editors' comments (where applicable) we provide additional requirements. Final acceptance of your manuscript is dependent on these requirements being met. We provide guidance below to help you prepare your revision.
Please submit your revised manuscript and required files (see below) no later than 7 days from today's (ie 09-Nov-2020) date. Note: the ScholarOne system will 'lock' if submission of the revision is attempted 7 or more days after the deadline. If you do not think you will be able to meet this deadline please contact the editorial office immediately.
Please note article processing charges apply to papers accepted for publication in Royal Society Open Science (https://royalsocietypublishing.org/rsos/charges). Charges will also apply to papers transferred to the journal from other Royal Society Publishing journals, as well as papers submitted as part of our collaboration with the Royal Society of Chemistry (https://royalsocietypublishing.org/rsos/chemistry). Fee waivers are available but must be requested when you submit your revision (https://royalsocietypublishing.org/rsos/waivers). The revised manuscript has now been reviewed by two referees. Both referees were satisfied that the revisions addressed the major comments raised in the initial review. Please address the very minor edits suggested by the referees and resubmit within 7 days for final acceptance.

Reviewer comments to Author:
Reviewer: 1 Comments to the Author(s) The manuscript has improved substantially. This new version avoids unnecessary claims of higher cognitive processes. I still fail to understand how the results found can be interpreted using prospect theory (page 19). The fact that decision-making is based on relative rather than on absolute metrics does not help us explain how 1 shrimp is better that 2. (by the way, a phenomenon cannot be akin to a theory, lines 233-236). Positive reinforcement is indeed a good and parsimonious explanation. The pending issue is whether the demonstration of reinforcement learning in cuttlefish is newsworthy. I do not think it is, but the science presented in the paper is solid.
Reviewer: 2 Comments to the Author(s) Comments attached ===PREPARING YOUR MANUSCRIPT=== Your revised paper should include the changes requested by the referees and Editors of your manuscript. You should provide two versions of this manuscript and both versions must be provided in an editable format:<ul><li>one version identifying all the changes that have been made (for instance, in coloured highlight, in bold text, or tracked changes);</li><li>a 'clean' version of the new manuscript that incorporates the changes made, but does not highlight them. This version will be used for typesetting.</li></ul> Please ensure that any equations included in the paper are editable text and not embedded images.
Please ensure that you include an acknowledgements' section before your reference list/bibliography. This should acknowledge anyone who assisted with your work, but does not qualify as an author per the guidelines at https://royalsociety.org/journals/ethicspolicies/openness/.
While not essential, it will speed up the preparation of your manuscript proof if you format your references/bibliography in Vancouver style (please see https://royalsociety.org/journals/authors/author-guidelines/#formatting). You should include DOIs for as many of the references as possible.
If you have been asked to revise the written English in your submission as a condition of publication, you must do so, and you are expected to provide evidence that you have received language editing support. The journal would prefer that you use a professional language editing service and provide a certificate of editing, but a signed letter from a colleague who is a native speaker of English is acceptable. Note the journal has arranged a number of discounts for authors using professional language editing services (https://royalsociety.org/journals/authors/benefits/language-editing/).

===PREPARING YOUR REVISION IN SCHOLARONE===
To revise your manuscript, log into https://mc.manuscriptcentral.com/rsos and enter your Author Centre -this may be accessed by clicking on "Author" in the dark toolbar at the top of the page (just below the journal name). You will find your manuscript listed under "Manuscripts with Decisions". Under "Actions", click on "Create a Revision".
Attach your point-by-point response to referees and Editors at Step 1 'View and respond to decision letter'. This document should be uploaded in an editable file type (.doc or .docx are preferred). This is essential.
Please ensure that you include a summary of your paper at Step 2 'Type, Title, & Abstract'. This should be no more than 100 words to explain to a non-scientific audience the key findings of your research. This will be included in a weekly highlights email circulated by the Royal Society press office to national UK, international, and scientific news outlets to promote your work.

At
Step 3 'File upload' you should include the following files: --Your revised manuscript in editable file format (.doc, .docx, or .tex preferred). You should upload two versions: 1) One version identifying all the changes that have been made (for instance, in coloured highlight, in bold text, or tracked changes); 2) A 'clean' version of the new manuscript that incorporates the changes made, but does not highlight them. --If you are requesting a discretionary waiver for the article processing charge, the waiver form must be included at this step.
--If you are providing image files for potential cover images, please upload these at this step, and inform the editorial office you have done so. You must hold the copyright to any image provided.
--A copy of your point-by-point response to referees and Editors. This will expedite the preparation of your proof.

At
Step 6 'Details & comments', you should review and respond to the queries on the electronic submission form. In particular, we would ask that you do the following: --Ensure that your data access statement meets the requirements at https://royalsociety.org/journals/authors/author-guidelines/#data. You should ensure that you cite the dataset in your reference list. If you have deposited data etc in the Dryad repository, please only include the 'For publication' link at this stage. You should remove the 'For review' link.
--If you are requesting an article processing charge waiver, you must select the relevant waiver option (if requesting a discretionary waiver, the form should have been uploaded at Step 3 'File upload' above).
--If you have uploaded ESM files, please ensure you follow the guidance at https://royalsociety.org/journals/authors/author-guidelines/#supplementary-material to include a suitable title and informative caption. An example of appropriate titling and captioning may be found at https://figshare.com/articles/Table_S2_from_Is_there_a_trade-off_between_peak_performance_and_performance_breadth_across_temperatures_for_aerobic_sc ope_in_teleost_fishes_/3843624.

At
Step 7 'Review & submit', you must view the PDF proof of the manuscript before you will be able to submit the revision. Note: if any parts of the electronic submission form have not been completed, these will be noted by red message boxes.

Decision letter (RSOS-201602.R1)
We hope you are keeping well at this difficult and unusual time. We continue to value your support of the journal in these challenging circumstances. If Royal Society Open Science can assist you at all, please don't hesitate to let us know at the email address below.
Dear Dr Chiao, It is a pleasure to accept your manuscript entitled "Learned valuation during forage decision making in cuttlefish" in its current form for publication in Royal Society Open Science.
You can expect to receive a proof of your article in the near future. Please contact the editorial office (openscience_proofs@royalsociety.org) and the production office (openscience@royalsociety.org) to let us know if you are likely to be away from e-mail contact --if you are going to be away, please nominate a co-author (if available) to manage the proofing process, and ensure they are copied into your email to the journal. Due to rapid publication and an extremely tight schedule, if comments are not received, your paper may experience a delay in publication.
Please see the Royal Society Publishing guidance on how you may share your accepted author manuscript at https://royalsociety.org/journals/ethics-policies/media-embargo/. I have now received two reviews of your manuscript entitled "Learned valuation during forage decision making in cuttlefish". Referee #1 expressed concerns about the appropriateness of invoking "less is more" and "loss aversion" arguments when interpreting the results. Upon careful reading of the manuscript, I felt that the description of the methods was not sufficiently clear to allow me to assess this. Referee #2 also pointed out that several aspects of the methods are unclear. Some of the details that are unclear after reading the manuscript make it impossible to assess the soundness of the interpretations that have been drawn from the results. As such, I am recommending that the manuscript be rejected with possibility to resubmit following substantial clarification of the methods. Response: We agree with Referee #1. We have toned down the "less is more" and "loss aversion" arguments in the Introduction and Discussion. We have also rewritten some of the Methods sections and now the description of the methods is much clear. We appreciate your help and giving us an opportunity to resubmit the manuscript. We hope that you find this revised manuscript much improved.
Should you choose to resubmit the manuscript, please address the following points, in addition to the very detailed points provided by referee #2. Response: We have addressed the following points and all the detailed points provided by Referee #2. 1. A schematic representation of the study setup would be usefulincluding the treatment orders, choices provided, rewards associated with each choice, and sample sizes (number of fish) and number of trials. Response: Thanks for the suggestion. We have included a supplementary table to show the schematic representation of the study setup.
2. Please be explicit about what total rewards were offered in each phase, including, for example, whether a single shrimp choice plus the additional reward of a "tiny" shrimp was equivalent to, or less than the reward experienced when choosing 2 shrimp in the baseline preference test (or were these trials all unrewarded?). This is critical for interpreting the change in preference in the third phase (preference for 1 shrimp over 2 shrimp) as the interpretation if 2 shrimp was never rewarded is very different than if 2 shrimp was rewarded with 2 shrimp in the baseline trial. If the former, then the study demonstrates associative learning, if the latter, then knowing how much the "tiny" shrimp was worth becomes critical. Response: Sorry for the confusion. We have added a supplementary table to summarize the reward or no reward associated with each choice. Except for the priming phase in Treatment group and Control group Ⅱ, the cuttlefish were not rewarded with any food, and this is to avoid the reinforcement of animal's decision when making a choice. In no cases cuttlefish were rewarded with the shrimp that was present in the apparatus once a choice was made. Thus, in the case of 1 vs. 2 choice, choosing the side with two shrimps were never rewarded. Indeed, in the principle of parsimony, the present study demonstrates associative learning, though other alternative interpretations may still exist. We have modified the Discussion in the revised manuscript accordingly.