"Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank" . . . . . . . . "Proceedings of the 7th International Conference on Language Resources and Evaluation (LREC 2010)" . "\u0160t\u011Bp\u00E1nek, Jan" . "RIV/00216208:11320/10:10078024" . . "P(GAP406/10/0875), P(GPP406/10/P193), P(LC536)" . "298632" . "European Language Resources Association" . . "2"^^ . . "2"^^ . "treebank; dependency; english; czech; prague; building; annotators; evaluation; ways"@en . "[6B63C42F887D]" . "Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank"@en . . "The paper presents several ways to measure and evaluate the annotation and annotators, proposed and used during the building of the Czech part of the Prague Czech-English Dependency Treebank. At first, the basic principles of the treebank annotation project are introduced (division to three layers: morphological, analytical and tectogrammatical). The main part of the paper describes in detail one of the important phases of the annotation process: three ways of evaluation of the annotators - inter-annotator agreement, error rate and performance. The measuring of the inter-annotator agreement is complicated by the fact that the data contain added and deleted nodes, making the alignment between annotations non-trivial. The error rate is measured by a set of automatic checking procedures that guard the validity of some invariants in the data. The performance of the annotators is measured by a booking web application. All three measures are later compared and related to each other."@en . . . "Mikulov\u00E1, Marie" . . "Valletta, Malta" . "The paper presents several ways to measure and evaluate the annotation and annotators, proposed and used during the building of the Czech part of the Prague Czech-English Dependency Treebank. At first, the basic principles of the treebank annotation project are introduced (division to three layers: morphological, analytical and tectogrammatical). The main part of the paper describes in detail one of the important phases of the annotation process: three ways of evaluation of the annotators - inter-annotator agreement, error rate and performance. The measuring of the inter-annotator agreement is complicated by the fact that the data contain added and deleted nodes, making the alignment between annotations non-trivial. The error rate is measured by a set of automatic checking procedures that guard the validity of some invariants in the data. The performance of the annotators is measured by a booking web application. All three measures are later compared and related to each other." . "Valletta, Malta" . . . . "Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank"@en . "RIV/00216208:11320/10:10078024!RIV11-GA0-11320___" . . "2010-05-17+02:00"^^ . . "11320" . . . "4"^^ . . . . "Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank" . . "2-9517408-6-7" .