About: Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank

Facets (new session)
Description
Metadata
Settings
- owl:sameAs
- Inference Rule:

About: Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank Goto Sponge NotDistinct Permalink

An Entity of Type : http://linked.opendata.cz/ontology/domain/vavai/Vysledek, within Data Space : linked.opendata.cz associated with source document(s)

Attributes	Values
rdf:type	skos:Concept http://linked.opendata.cz/ontology/domain/vavai/Vysledek
Description	The paper presents several ways to measure and evaluate the annotation and annotators, proposed and used during the building of the Czech part of the Prague Czech-English Dependency Treebank. At first, the basic principles of the treebank annotation project are introduced (division to three layers: morphological, analytical and tectogrammatical). The main part of the paper describes in detail one of the important phases of the annotation process: three ways of evaluation of the annotators - inter-annotator agreement, error rate and performance. The measuring of the inter-annotator agreement is complicated by the fact that the data contain added and deleted nodes, making the alignment between annotations non-trivial. The error rate is measured by a set of automatic checking procedures that guard the validity of some invariants in the data. The performance of the annotators is measured by a booking web application. All three measures are later compared and related to each other. The paper presents several ways to measure and evaluate the annotation and annotators, proposed and used during the building of the Czech part of the Prague Czech-English Dependency Treebank. At first, the basic principles of the treebank annotation project are introduced (division to three layers: morphological, analytical and tectogrammatical). The main part of the paper describes in detail one of the important phases of the annotation process: three ways of evaluation of the annotators - inter-annotator agreement, error rate and performance. The measuring of the inter-annotator agreement is complicated by the fact that the data contain added and deleted nodes, making the alignment between annotations non-trivial. The error rate is measured by a set of automatic checking procedures that guard the validity of some invariants in the data. The performance of the annotators is measured by a booking web application. All three measures are later compared and related to each other. (en)
Title	Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank (en)
skos:prefLabel	Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank (en)
skos:notation	RIV/00216208:11320/10:10078024!RIV11-GA0-11320___
http://linked.open...avai/riv/aktivita	P
http://linked.open...avai/riv/aktivity	P(GAP406/10/0875), P(GPP406/10/P193), P(LC536)
http://linked.open...vai/riv/dodaniDat	2011
http://linked.open...aciTvurceVysledku	Štěpánek, Jan Mikulová, Marie
http://linked.open.../riv/druhVysledku	D - Článek ve sborníku
http://linked.open...iv/duvernostUdaju	S - Úplné a pravdivé údaje nepodléhající ochraně podle zvláštních právních předpisů
http://linked.open...titaPredkladatele	Univerzita Karlova v Praze / Matematicko-fyzikální fakulta
http://linked.open...dnocenehoVysledku	298632
http://linked.open...ai/riv/idVysledku	RIV/00216208:11320/10:10078024
http://linked.open...riv/jazykVysledku	eng - angličtina
http://linked.open.../riv/klicovaSlova	treebank; dependency; english; czech; prague; building; annotators; evaluation; ways (en)
http://linked.open.../riv/klicoveSlovo	english building dependency evaluation prague annotators czech treebank ways
http://linked.open...ontrolniKodProRIV	[6B63C42F887D]
http://linked.open...v/mistoKonaniAkce	Valletta, Malta
http://linked.open...i/riv/mistoVydani	Valletta, Malta
http://linked.open...i/riv/nazevZdroje	Proceedings of the 7th International Conference on Language Resources and Evaluation (LREC 2010)
http://linked.open...in/vavai/riv/obor	AI
http://linked.open...ichTvurcuVysledku	2 (xsd:int)
http://linked.open...cetTvurcuVysledku	2 (xsd:int)
http://linked.open...vavai/riv/projekt	Integrated center for natural language processing Computational Linguistics: Explicit description of language and annotated data focused on Czech Tools for Revision and Tectogrammatical Annotation of a Czech Dependency Treebank
http://linked.open...UplatneniVysledku	2010
http://linked.open...iv/tvurceVysledku	Štěpánek, Jan Mikulová, Marie
http://linked.open...vavai/riv/typAkce	WRD - Světová
http://linked.open.../riv/zahajeniAkce	2010-05-17 (xsd:date)
number of pages	4 (xsd:int)
http://purl.org/ne...btex#hasPublisher	European Language Resources Association
https://schema.org/isbn	2-9517408-6-7
http://localhost/t...ganizacniJednotka	11320
is http://linked.open...avai/riv/vysledek of	Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank

Faceted Search & Find service v1.16.118 as of Jun 21 2024

Alternative Linked Data Documents: ODE Content Formats:

RDF

ODATA

Microdata

About

OpenLink Virtuoso version 07.20.3240 as of Jun 21 2024, on Linux (x86_64-pc-linux-gnu), Single-Server Edition (126 GB total memory, 47 GB memory in use)
Data on this page belongs to its respective rights holders.
Virtuoso Faceted Browser Copyright © 2009-2024 OpenLink Software