The establishment and evaluation of annotated resources as well as automatic annotation systems have growing interest in linguistics and NLP. However, evaluation methods are not always well mastered and used, particularly in linguistics, where the sequential nature of textual data requires specific methods. In this article, we first analyze the notions of corpus and annotation in their multiple forms, then focus on the results of the evaluation questions. In particular, we emphasize the difference in nature between the evaluation of an annotation system and the evaluation of manually multi-annotated resources. We provide an overview of existing evaluation tools and their possible use for each of these two cases.