Lameness is an important welfare problem in modern dairy farms. In order to support lameness control in dairy farms, several concepts to develop a reliable automatic lameness assessment systems (ALAS ) have been undertaken. Golden standards methods used for ALAS development are discussed based on a literature review. 16 out of 18 articles related with automatic lameness assessment systems used locomotion scores as gold standard. Main advantages of locomotion scores as gold standard lie on their practical application. Locomotion scores are easy and cheap to use in practice. Disadvantages of locomotion scores as gold standard are related with validity and repeatability of the method. Locomotion scores as tool for pain and hoof lesions (potential source of pain) seems limited because, cows seems to present an important tolerance to pain and locomotion is also impaired by the practical conditions in which it is assessed. Repeatability for locomotion assessment showed a large variation between and within observers. The main factor in the variation of repeatability seems to be the subjectivity associated to locomotion scores. Poor validity and repeatability hinders the interpretation of what is finally being assessed by LSs and ultimately by ALAS. Hoof and painful lesions are also used as gold standard for ALAS development. As LS diagnosis of hoof and painful lesions are also subjective and may present poor repeatability. In conclusion, currently there is not an optimal gold standard for the development of ALAS.