Automated detection of lameness in sheep using machine learning approaches: novel insights into behavioural differences among lame and non-lame sheep

Lameness in sheep is the biggest cause of concern regarding poor health and welfare among sheep-producing countries. Best practice for lameness relies on rapid treatment, yet there are no objective measures of lameness detection. Accelerometers and gyroscopes have been widely used in human activity studies and their use is becoming increasingly common in livestock. In this study, we used 23 datasets (10 non-lame and 13 lame sheep) from an accelerometer- and gyroscope-based ear sensor with a sampling frequency of 16 Hz to develop and compare algorithms that can differentiate lameness within three different activities (walking, standing and lying). We show for the first time that features extracted from accelerometer and gyroscope signals can differentiate between lame and non-lame sheep while standing, walking and lying. The random forest algorithm performed best for classifying lameness with an accuracy of 84.91% within lying, 81.15% within standing and 76.83% within walking and overall correctly classified over 80% sheep within activities. Both accelerometer- and gyroscope-based features ranked among the top 10 features for classification. Our results suggest that novel behavioural differences between lame and non-lame sheep across all three activities could be used to develop an automated system for lameness detection.


3
Please submit a copy of your revised paper before 19-Sep-2019. Please note that the revision deadline will expire at 00.00am on this date. If we do not hear from you within this time then it will be assumed that the paper has been withdrawn. In exceptional circumstances, extensions may be possible if agreed with the Editorial Office in advance. We do not allow multiple rounds of revision so we urge you to make every effort to fully address all of the comments at this stage. If deemed necessary by the Editors, your manuscript will be sent back to one or more of the original reviewers for assessment. If the original reviewers are not available, we may invite new reviewers.
To revise your manuscript, log into http://mc.manuscriptcentral.com/rsos and enter your Author Centre, where you will find your manuscript title listed under "Manuscripts with Decisions." Under "Actions," click on "Create a Revision." Your manuscript number has been appended to denote a revision. Revise your manuscript and upload a new version through your Author Centre.
When submitting your revised manuscript, you must respond to the comments made by the referees and upload a file "Response to Referees" in "Section 6 -File Upload". Please use this to document how you have responded to the comments, and the adjustments you have made. In order to expedite the processing of the revised manuscript, please be as specific as possible in your response.
In addition to addressing all of the reviewers' and editor's comments please also ensure that your revised manuscript contains the following sections as appropriate before the reference list: • Ethics statement (if applicable) If your study uses humans or animals please include details of the ethical approval received, including the name of the committee that granted approval. For human studies please also detail whether informed consent was obtained. For field studies on animals please include details of all permissions, licences and/or approvals granted to carry out the fieldwork.
• Data accessibility It is a condition of publication that all supporting data are made available either as supplementary information or preferably in a suitable permanent repository. The data accessibility section should state where the article's supporting data can be accessed. This section should also include details, where possible of where to access other relevant research materials such as statistical tools, protocols, software etc can be accessed. If the data have been deposited in an external repository this section should list the database, accession number and link to the DOI for all data from the article that have been made publicly available. Data sets that have been deposited in an external repository and have a DOI should also be appropriately cited in the manuscript and included in the reference list.
If you wish to submit your supporting data or code to Dryad (http://datadryad.org/), or modify your current submission to dryad, please use the following link: http://datadryad.org/submit?journalID=RSOS&manu=RSOS-190824 • Competing interests Please declare any financial or non-financial competing interests, or state that you have no competing interests.
• Authors' contributions All submissions, other than those with a single author, must include an Authors' Contributions section which individually lists the specific contribution of each author. The list of Authors should meet all of the following criteria; 1) substantial contributions to conception and design, or acquisition of data, or analysis and interpretation of data; 2) drafting the article or revising it critically for important intellectual content; and 3) final approval of the version to be published.
All contributors who do not meet all of these criteria should be included in the acknowledgements.
We suggest the following format: AB carried out the molecular lab work, participated in data analysis, carried out sequence alignments, participated in the design of the study and drafted the manuscript; CD carried out the statistical analyses; EF collected field data; GH conceived of the study, designed the study, coordinated the study and helped draft the manuscript. All authors gave final approval for publication.
• Acknowledgements Please acknowledge anyone who contributed to the study but did not meet the authorship criteria.
• Funding statement Please list the source of funding for each author.
Once again, thank you for submitting your manuscript to Royal Society Open Science and I look forward to receiving your revision. If you have any questions at all, please do not hesitate to get in touch. Both reviewers see substantial merit in the paper and would like to see it published, but request additional details so that the methods and results are presented in a more complete manner. We look forward to a revised article in due course.
Reviewers' Comments to Author: Reviewer: 1 Comments to the Author(s) See attached pdf for my comments.
Reviewer: 2 Comments to the Author(s) The manuscript was enjoyable and certainly an interesting and important concept. The results are important for animal welfare scientists, the sheep-producing community and for those interested in precision livestock technology. I would like to see this published and hope that the comments in the attached file will help to strengthen the paper.
Author's Response to Decision Letter for (RSOS-190824

Do you have any ethical concerns with this paper? No
Have you any concerns about statistical analyses in this paper? No

Recommendation? Accept as is
Comments to the Author(s) I am very pleased with these revisions clarifying the generalizability of the model. I think the paper can be accepted in its current form.

Review form: Reviewer 2
Is the manuscript scientifically sound in its present form? Yes

Are the interpretations and conclusions justified by the results? Yes
Is the language acceptable? Yes 6 Do you have any ethical concerns with this paper? No

Recommendation?
Accept as is

Comments to the Author(s)
The authors have done a great job in addressing the comments and updating the manuscript. The paper will be of interest to many in academia and industry and I look forward to seeing it published.

29-Nov-2019
Dear Dr Kaler, It is a pleasure to accept your manuscript entitled "Automated detection of lameness in sheep using machine learning approaches: novel insights into behavioural differences among lame and non-lame sheep" in its current form for publication in Royal Society Open Science. The comments of the reviewer(s) who reviewed your manuscript are included at the foot of this letter.
Please ensure that you send to the editorial office an editable version of your accepted manuscript, and individual files for each figure and table included in your manuscript. You can send these in a zip folder if more convenient. Failure to provide these files may delay the processing of your proof. You may disregard this request if you have already provided these files to the editorial office.
You can expect to receive a proof of your article in the near future. Please contact the editorial office (openscience_proofs@royalsociety.org) and the production office (openscience@royalsociety.org) to let us know if you are likely to be away from e-mail contact --if you are going to be away, please nominate a co-author (if available) to manage the proofing process, and ensure they are copied into your email to the journal. Due to rapid publication and an extremely tight schedule, if comments are not received, your paper may experience a delay in publication.
Please see the Royal Society Publishing guidance on how you may share your accepted author manuscript at https://royalsociety.org/journals/ethics-policies/media-embargo/. I am very pleased with these revisions clarifying the generalizability of the model. I think the paper can be accepted in its current form.
Reviewer: 2 Comments to the Author(s) The authors have done a great job in addressing the comments and updating the manuscript. The paper will be of interest to many in academia and industry and I look forward to seeing it published.
Follow Royal Society Publishing on Twitter: @RSocPublishing Follow Royal Society Publishing on Facebook: https://www.facebook.com/RoyalSocietyPublishing.FanPage/ Read Royal Society Publishing's blog: https://blogs.royalsociety.org/publishing/ A summary of my review of "Automated detection of lameness in sheep using machine learning approaches: novel insights into behavioural differences among lame and non-lame sheep": In this paper, the authors use machine learning to detect lameness in sheep from body-worn sensor data. Compared to previous literature which has focused on walking, here they find that useful information needed for lameness classification is present when standing and lying in addition to walking. The paper is clearly written, choice of the model used is justified and the methodology is clearly outlined. It is good to see that the authors have conducted crossvalidation. However, my main suggested revision is to (i) directly quantify how balanced the dataset is and (ii) to clearly quantify how much leakage there is between the training and test set. This is essential for drawing the right conclusions from the modeling results and for evaluating the generalizability of the model developed here -if researchers want to use them on a new dataset. Once my suggestion (detailed below) is addressed and if the code and data are made publicly available, the paper can be accepted in my opinion.
My suggestions for revision are: 1) Clear details about about how balanced the train and test data are is missing and is necessary to draw the right conclusions from the modeling results: i) % of train and test data from each sheep, iii) % of train and test sheep-wise data that are from a given activity i.e. walking vs lying vs standing, iv) % of misclassifications by the model that are for a given sheep and v) how many sheep were included in both train and test data (leakage). Without this information it is difficult to interpret the success of the model. For example, if all the misclassifications are for the data of 4 sheep then the actual success of the model is 9 out of 13 sheep. Or if a vast majority of the datapoints correspond to standing then the higher accuracy for standing classification is less surprising. Adding these simple details about the data would help the reader gauge how generalizable this model will be to their own dataset. The authors must also add a paragraph in the discussion section on how they think their model will generalize to new data. 2) The caption in figure 6 has typos: "lying, standing and lying" and "that was predicted as not been lame". Please read the manuscript carefully once more for typos. 3) I am unable to access the data on dryad with the link provided.

Automated detection of lameness in sheep using machine learning approaches: novel insights into behavioural differences among lame and non-lame sheep
Many thanks for the opportunity to review this manuscript, which is timely in terms of both animal welfare and application of precision livestock technology. This paper presents a method to automatically detect the lameness in sheep using the data collected from an accelerometer and gyroscope ear sensor. The machine learning techniques are employed to classify the sheep into the lameness and non-lameness within three activities (standing, walking and lying), where the accelerometer and gyroscope based Features, time and frequency features are used in two-phase classifications. Overall, the paper is well structured, but lacks descriptions of some details of the method, especially that the key Hyperparameters in each classifier are not clarified. The experimental results are presented well. General comments and more detailed points of the main concerns for this manuscript are as follows: General comments:  The structure and grammar of sentences needs proofreading and correcting throughout. There are some places where the tense changes in the sentence and some sentences with extra words where they are not needed e.g. "with 12 in the 2016".  The introduction could benefit from more detail. The order of the introduction is also a little confusing. It feels like there are 5 main points to make: food security, the use of sheep, lameness, early detection, technology. The structure jumps a little between these ideas and makes it slightly difficult for the reader to follow.  Please add your hypotheses and predictions to the end of the introduction.  The Study site and animals section needs more detail. Please see the ARRIVE guidelines for some pointers of what could be added (https://www.nc3rs.org.uk/sites/default/files/documents/Guidelines/NC3Rs%20ARRIV E%20Guidelines%202013.pdf)  In section 3, the authors mentioned that two-phase is composed of activities classification (first phase) and lameness classification (send phase), please highlight the novelty of the method presented in this manuscript compared to the previous work in [13].  As one of data pre-processing steps, the dynamic component of the acceleration and gyroscope sensors was removed. It is desirable that the authors provide any other processing that was applied to de-noise the raw data and balance the training data?  The authors have presented results to compare different classifiers according to the different number of features. The factor of features and their dimensionalities matters to the classification performance. However, it is not the only factor that could significantly affect the classifications. In the manuscript, some important details for each classifier need to be clarified. These details are critical which should be considered in the comparison of classifications. Has each classifier been optimized by hyperparameters tuning? It is desirable that the authors could provide more information related to classifiers. E.g. for RF, what are these parameters: the number of trees and the depth of each tree in the forest, the number of features to consider when looking for the best split. Kernel selection in the SVM. For the NN, what hyperparameters (number of layers, activation function, learning rate) are selected in the experiments. For the Adaboost, what are the base learner, the number of base learner and learning rate? For the KNN, what is the number of K, the distance metric and normalization adopted in the training?
Detailed comments: Page 3 Line 13: As there are only 3 behaviours to describe, it would be useful to see the definitions within the paper.