Social value orientation and attachment: a replication and extension of Van Lange et al. (1997)

We report a replication and extension of a finding from Studies 1 and 2 of Van Lange et al.'s influential paper (Van Lange et al. 1997 J. Pers. Soc. Psychol. 73, 733–746. (doi:10.1037/0022-3514.73.4.733)), which showed an association between Social Value Orientation (SVO) and attachment security. We report a close replication but with measures of attachment that are considered superior in comparison to measures used by Van Lange et al., due to subsequent psychometric improvements. Psychometric analyses indeed showed that our attachment measures were reliable and valid, demonstrating theoretically predicted associations with other outcomes. With a sample (N = 879) sufficiently large to detect d = 0.19 (and larger than the original N = 573), we failed to replicate the effect. Based on the available evidence, we interpret as there being no evidence for the link between attachment security and Social Value Orientation, but further replication research that uses solid measures and large samples can provide more definite conclusions about the association between attachment and SVO.

When submitting your revised manuscript, you must respond to the comments made by the referees and upload a file "Response to Referees" in the "File Upload" step. Please use this to document how you have responded to the comments, and the adjustments you have made. In order to expedite the processing of the revised manuscript, please be as specific as possible in your response.
Once again, thank you for submitting your manuscript to Royal Society Open Science and I look forward to receiving your revision. If you have any questions at all, please do not hesitate to get in touch. Full author guidelines may be found at http://rsos.royalsocietypublishing.org/page/replication-studies#AuthorsGuidance.
Kind regards, Professor Chris Chambers Royal Society Open Science openscience@royalsociety.org on behalf of Professor Chris Chambers (Registered Reports Editor, Royal Society Open Science) openscience@royalsociety.org Associate Editor Comments to Author (Professor Chris Chambers): Two expert reviewers have now appraised the manuscript. As you will see, both are positive about the submission but note several areas requiring attention. The most substantial concern (Reviewer #1) is the need for greater detail about the analyses and certain aspects of the instruments. In addition to this major issue, both reviewers offer a range of constructive suggestions for clarifying and justifying specific arguments.
Comments to Author: Reviewer: 1 Comments to the Author(s) Please see the attached file.

Reviewer: 2
Comments to the Author(s) This is a sound replication of previous research by Van  on social value orientation and adult attachment. I have two minor suggestions that I think would help improve the paper. First, the authors seem to be implying that the large number of citations that the original article received is a marker of the paper's influence. But my sense, from examining some of the articles that have cited the original is that those citations are not for the attachment-SVO findings/claims, but rather simply a citation for the SVO idea itself and some of the methods that were used to study it. I think it is potentially misleading to imply that the attachment-SVO findings have been influential in the field. Second, on page 5, I would caution against suggesting that adult attachment patterns are a function of "early" experience. The statement implicitly suggests that later experiences are not relevant. I doubt the authors believe that, but that is one way to interpret the use of the term "early" rather than, say, "prior" or "a history of interpersonal experiences." I think the methods and analytic strategies are sound. I'm glad the sample size is relatively large.
of " has been accepted in principle for publication in Royal Society Open Science. The reviewers' and editors' comments are included at the end of this email.
You may now progress to Stage 2 and complete the study as approved.
You must now register your approved protocol on the Open Science Framework (https://osf.io/rr), either publicly or privately under embargo until submission of the Stage 2 manuscript. Please note that a time-stamped, independent registration of the protocol is mandatory under journal policy, and manuscripts that do not conform to this requirement cannot be considered at Stage 2. The protocol should be registered unchanged from its current approved state. Please include a URL to the protocol in your Stage 2 manuscript.
Following completion of your study, we invite you to resubmit your paper for peer review as a Stage 2 Replication. Please note that your manuscript can still be rejected for publication at Stage 2 if the Editors consider any of the following conditions to be met: • The Introduction and methods deviated from the approved Stage 1 submission (required). • The authors' conclusions were not considered justified given the data.
We encourage you to read the complete guidelines for authors concerning Stage 2 submissions at: http://rsos.royalsocietypublishing.org/page/replication-studies#AuthorsGuidance. Please especially note the requirements for data sharing and that withdrawing your manuscript will result in publication of a Withdrawn Registration.
Once again, thank you for submitting your manuscript to Royal Society Open Science and I look forward to receiving your Stage 2 submission. If you have any questions at all, please do not hesitate to get in touch. We look forward to hearing from you shortly with the anticipated submission date for your stage two manuscript.

Recommendation? Accept with minor revision
Comments to the Author(s) I have one minor and one major point that I would like the authors to adress before I can recommend the manuscript for publication.

Minor
The authors should explicate the number of individuals that could not be classified. Also there is currently no description of the criterion/cutoff used for the classification.

Major
Following my request from the first stage review, the authors have added a footnote on the availability of better SVO measures. However, I do not think that this should be put in a footnote, it deserves to be part of the main body of the discussion. First, the very philosophy of the authors (which I endorse!) is that replication should make use of the development of better measures. To be consistent with this thinking, the authors should not only stress to what extent their paper meets this standard but also point to their failures of meeting it (although it was clearly not a fault as new SVO instruments were freshly developed when data collection commenced). This does not weaken the paper but instead makes it more consistent and convincing. Second, the authors' argument that their SVO scale was highly consistent based on KR20 is at least difficult. On the one hand, this statistic rests on an artifical dichotomization of paricipants' answers. On the other hand, it is not surprising to get a high KR20s given that the triple dominance scales essentially confront participants with three highly similar choices (Murphy & Ackermann, 2014, PERS SOC PSYCHOL REV). Third (and related), the very point about newly developed SVO scales (e.g., Murphy, Ackermann, & Handgraaf, 2011, JUDGM DECIS MAK) is the insight that gradual differences in SVO can matter much more than nominal classifications -classifications can even conceal meaningful differences due to SVO (for illustrations of this important point, see Bieleke, Gollwitzer, Oettingen, & Fischbacher, 2017, J BEHAV DECIS MAKING; Fiedler, Glöckner, Nicklisch, & Dickert, 2013, ORGAN BEHAV HUM DEC). To me, this insight seems at least as important as the new developments regarding attachment scales stressed in the present paper. Adressing this point more thoroughly would make the paper a much more useful contribution for researchers who care about SVO. Moreover, it provides a promising avenue for future research -which, for instance, might examine whether gradual SVO differences relate more robustly to attachment.

Review form: Reviewer 2
Do you have any ethical concerns with this paper? No

Recommendation? Accept as is
Comments to the Author(s) I believe the authors have done an excellent job at analyzing the data and providing a fair and rigorous test of the key idea (i.e., that there is an association between attachment security and SVO).
I also believe the authors have summarized their findings well, discussed them in valuable ways, and have been appropriately cautionary about what has and has not been learned from this research.
In short, I think this manuscript is ready to be published. The reviewers and Subject Editor have recommended publication, but also suggest some minor revisions to your manuscript. Therefore, I invite you to respond to the comments and revise your manuscript.
Please also ensure that all the below editorial sections are included where appropriate (a nonexhaustive example is included in an attachment): • Ethics statement If your study uses humans or animals please include details of the ethical approval received, including the name of the committee that granted approval. For human studies please also detail whether informed consent was obtained. For field studies on animals please include details of all permissions, licences and/or approvals granted to carry out the fieldwork.
• Data accessibility It is a condition of publication that all supporting data are made available either as supplementary information or preferably in a suitable permanent repository. The data accessibility section should state where the article's supporting data can be accessed. This section should also include details, where possible of where to access other relevant research materials such as statistical tools, protocols, software etc can be accessed. If the data has been deposited in an external repository this section should list the database, accession number and link to the DOI for all data from the article that has been made publicly available. Data sets that have been deposited in an external repository and have a DOI should also be appropriately cited in the manuscript and included in the reference list.
If you wish to submit your supporting data or code to Dryad (http://datadryad.org/), or modify your current submission to dryad, please use the following link: http://datadryad.org/submit?journalID=RSOS&manu=RSOS-181575.R2 • Competing interests Please declare any financial or non-financial competing interests, or state that you have no competing interests.
• Authors' contributions All submissions, other than those with a single author, must include an Authors' Contributions section which individually lists the specific contribution of each author. The list of Authors should meet all of the following criteria; 1) substantial contributions to conception and design, or acquisition of data, or analysis and interpretation of data; 2) drafting the article or revising it critically for important intellectual content; and 3) final approval of the version to be published.
All contributors who do not meet all of these criteria should be included in the acknowledgements.
We suggest the following format: AB carried out the molecular lab work, participated in data analysis, carried out sequence alignments, participated in the design of the study and drafted the manuscript; CD carried out the statistical analyses; EF collected field data; GH conceived of the study, designed the study, coordinated the study and helped draft the manuscript. All authors gave final approval for publication.
• Acknowledgements Please acknowledge anyone who contributed to the study but did not meet the authorship criteria.
• Funding statement Please list the source of funding for each author.
Because the schedule for publication is very tight, it is a condition of publication that you submit the revised version of your manuscript within 7 days (i.e. by the 15-Feb-2019). If you do not think you will be able to meet this date please let me know immediately.
To revise your manuscript, log into https://mc.manuscriptcentral.com/rsos and enter your Author Centre, where you will find your manuscript title listed under "Manuscripts with Decisions". Under "Actions," click on "Create a Revision." You will be unable to make your revisions on the originally submitted version of the manuscript. Instead, revise your manuscript and upload a new version through your Author Centre.
When submitting your revised manuscript, you will be able to respond to the comments made by the referees and upload a file "Response to Referees" in "Section 6 -File Upload". You can use this to document any changes you make to the original manuscript. In order to expedite the processing of the revised manuscript, please be as specific as possible in your response to the referees.
When uploading your revised files please make sure that you have: 1) A text file of the manuscript (tex, txt, rtf, docx or doc), references, tables (including captions) and figure captions. Do not upload a PDF as your "Main Document". 2) A separate electronic file of each figure (EPS or print-quality PDF preferred (either format should be produced directly from original creation package), or original software format) 3) Included a 100 word media summary of your paper when requested at submission. Please ensure you have entered correct contact details (email, institution and telephone) in your user account 4) Included the raw data to support the claims made in your paper. You can either include your data as electronic supplementary material or upload to a repository and include the relevant DOI within your manuscript 5) Included your supplementary files in a format you are happy with (no line numbers, Vancouver referencing, track changes removed etc) as these files will NOT be edited in production Once again, thank you for submitting your manuscript to Royal Society Open Science and I look forward to receiving your revision. If you have any questions at all, please do not hesitate to get in touch.
Kind regards, Professor Chris Chambers Royal Society Open Science openscience@royalsociety.org Associate Editor Comments to Author (Professor Chris Chambers): Associate Editor: 1 Comments to the Author: The manuscript was returned to the two expert reviewers who assessed it at Stage 1. Both are positive about the submission, with Reviewer 1 recommending some elaboration of the Discussion, and Reviewer 2 recommending immediate acceptance. Please attend carefully to the comments of Reviewer 1. Provided these concerns are fully addressed in revision, full acceptance should be forthcoming without requiring further in-depth review.
Reviewers' comments to Author: Reviewer: 1 Comments to the Author(s) I have one minor and one major point that I would like the authors to adress before I can recommend the manuscript for publication.

Minor
The authors should explicate the number of individuals that could not be classified. Also there is currently no description of the criterion/cutoff used for the classification.

Major
Following my request from the first stage review, the authors have added a footnote on the availability of better SVO measures. However, I do not think that this should be put in a footnote, it deserves to be part of the main body of the discussion. First, the very philosophy of the authors (which I endorse!) is that replication should make use of the development of better measures. To be consistent with this thinking, the authors should not only stress to what extent their paper meets this standard but also point to their failures of meeting it (although it was clearly not a fault as new SVO instruments were freshly developed when data collection commenced). This does not weaken the paper but instead makes it more consistent and convincing. Second, the authors' argument that their SVO scale was highly consistent based on KR20 is at least difficult. On the one hand, this statistic rests on an artifical dichotomization of paricipants' answers. On the other hand, it is not surprising to get a high KR20s given that the triple dominance scales essentially confront participants with three highly similar choices ( To me, this insight seems at least as important as the new developments regarding attachment scales stressed in the present paper. Adressing this point more thoroughly would make the paper a much more useful contribution for researchers who care about SVO. Moreover, it provides a promising avenue for future research -which, for instance, might examine whether gradual SVO differences relate more robustly to attachment.

Reviewer: 2
Comments to the Author(s) I believe the authors have done an excellent job at analyzing the data and providing a fair and rigorous test of the key idea (i.e., that there is an association between attachment security and SVO).
I also believe the authors have summarized their findings well, discussed them in valuable ways, and have been appropriately cautionary about what has and has not been learned from this research.
In short, I think this manuscript is ready to be published.  It is a pleasure to accept your Stage 2 Replication entitled "Social Value Orientation and Sunday, October 28, 2018 5:11 PM The authors describe four waves of data collection aiming at a replication of van Lange et al.'s (1997; Studies 1 and 2) finding that social value orientation (SVO) is associated with adult attachment style. It is a Stage 1 replication study with the blinded results of an already completed data collection. I find this research timely and interesting, although I had some remarks that I would like the authors to address. Below are my thoughts and remarks regarding the different reviewer questions.
1. Stage 1 Primary Criterion #1: Whether the authors provide a sufficiently clear and detailed description of the methods for another researcher to closely replicate the proposed experimental procedures and analysis pipeline, and to prevent undisclosed flexibility in the experimental procedures or analysis pipeline. * The authors report the measures included in all four waves of data collection. They should make explicit whether they used the exact same triple-dominance scale as van Lange et al. to measure SVO. This seems to be the case but it should be explicit and the resulting effective number of participants (including non-classifiable participants) should be mentioned.
* The authors want to make their data and analysis syntax publicly available. However, I cannot check this because the link has been redacted for review.
* Regarding the analysis pipeline, the central piece of information is missing: How will the association between SVO and attachment style be tested? Will the authors use the same approach as van Lange et al. * I have one major issue with this research: While the authors focus on better measures of attachment, they seem to have missed crucial developments regarding the assessment of SVO (see Murphy, Ackermann, & Handgraaf, 2011;Murphy & Ackermann, 2013). The scale used by van Lange et al. has meanwhile been shown to be clearly dominated by continuous SVO scales, and the use of nominal categories is commonly discouraged for theoretical reasons (e.g., Fiedler, Glöckner, Nicklisch, & Dickert, 2013). This said, I want to stress that I like the authors' attempt to test whether an important empirical finding still holds when improved measures are used, rather than literally repeating the same experiment. However, this approach would have required to measure SVO in a modern and empirically sound way as well. This aspect reduces my enthusiasm about the described experiment. Since the experiment has already been conducted, the authors should at least address this limitation later in the discussion of their manuscript.
* To re-iterate, I feel that the analysis pipeline is not described in sufficient detail to determine whether or not it is sound.
-1-Appendix A Sunday, October 28, 2018 5:11 PM 5. Stage 1 Secondary Criterion #3: Whether the authors have considered sufficient outcome-neutral conditions (e.g. absence of floor or ceiling effects; positive controls; other quality checks) for ensuring that the results obtained are able to test the stated hypotheses.
* It would have desirable to have van Lange et al.'s original instruments in these experiments as well. This way, the authors could have tested whether differences in the findings reflect differences in the applied measures or result from other differences in the experimental protocol. * Otherwise, the authors' approach seems sound to me.