Observer bias in applied ethology: can we believe what we score, if we score what we believe?

Rating animal behaviour and condition is the methodological cornerstone of applied ethology. As blinding observers is not common and often not feasible, the data collection staff is usually aware of relevant information about the animals they observe. This information may evoke expectations about the rating outcome which in turn may lead to conscious or unconscious biases in the data, i.e. expectation bias. In order to start documenting the extent to which this type of observer bias may pose a problem in applied ethology, we investigated whether observer expectations influenced subjective scoring methods during a class practicum. Veterinary students (n=157) were briefly trained in (a) assessing laying hen behaviour using Qualitative Behaviour Assessment (QBA), (b) scoring the degree of panting to assess heat stress in cattle, and (c) recording negative and positive interactions between pigs. The students were divided into two groups and applied these methods in three experiments. During these experiments they were shown duplicated video recordings of the same animals: the original and a slightly modified version (to prevent recognition at second viewing). Prior to scoring they were either told correct or false information about the conditions in which the animals had been filmed. One group was given false information for the modified recording and correct information for the original recording, and vice versa for the other group. The false information reflected plausible study scenarios in ethology and aimed to create expectations about the outcome. As in reality they scored the identical behaviour twice, the difference in scores when given true versus false information reflects expectation bias. In all experiments there was evidence of expectation bias: the QBA indicated more positive and fewer negative emotions when told that the laying hens were from an organic instead of a conventional farm, the cattle panting score was higher when told that the ambient temperature was 5°C higher than in reality, and the ratio of positive to negative interactions was higher when told that the observed pigs had been selected for high social breeding value. Although veterinary students may not be representative of practicing ethologists, these findings do indicate that observer bias could influence subjective scores of animal behaviour and welfare. We discuss the implications of these findings in the field of applied ethology and suggest possible solutions.