About: Quiz-Based Evaluation of Machine Translation

Facets (new session)
Description
Metadata
Settings
- owl:sameAs
- Inference Rule:

About: Quiz-Based Evaluation of Machine Translation Goto Sponge NotDistinct Permalink

An Entity of Type : http://linked.opendata.cz/ontology/domain/vavai/Vysledek, within Data Space : linked.opendata.cz associated with source document(s)

Attributes	Values
rdf:type	skos:Concept http://linked.opendata.cz/ontology/domain/vavai/Vysledek
Description	This paper proposes a new method of manual evaluation for statistical machine translation, the so-called quiz-based evaluation, estimating whether people are able to extract information from machine-translated texts reliably. We apply the method to two commercial and two experimental MT systems that participated in WMT 2010 in English-to-Czech translation. We report inter-annotator agreement for the evaluation as well as the outcomes of the individual systems. The quiz-based evaluation suggests rather different ranking of the systems compared to the WMT 2010 manual and automatic metrics. We also see that overall, MT quality is becoming acceptable for obtaining information from the text: about 80% of questions can be answered correctly given only machine-translated text. This paper proposes a new method of manual evaluation for statistical machine translation, the so-called quiz-based evaluation, estimating whether people are able to extract information from machine-translated texts reliably. We apply the method to two commercial and two experimental MT systems that participated in WMT 2010 in English-to-Czech translation. We report inter-annotator agreement for the evaluation as well as the outcomes of the individual systems. The quiz-based evaluation suggests rather different ranking of the systems compared to the WMT 2010 manual and automatic metrics. We also see that overall, MT quality is becoming acceptable for obtaining information from the text: about 80% of questions can be answered correctly given only machine-translated text. (en)
Title	Quiz-Based Evaluation of Machine Translation Quiz-Based Evaluation of Machine Translation (en)
skos:prefLabel	Quiz-Based Evaluation of Machine Translation Quiz-Based Evaluation of Machine Translation (en)
skos:notation	RIV/00216208:11320/11:10107804!RIV12-GA0-11320___
http://linked.open...avai/riv/aktivita	P
http://linked.open...avai/riv/aktivity	P(7E09003), P(GAP406/11/1499), P(GPP406/10/P259)
http://linked.open...iv/cisloPeriodika	1
http://linked.open...vai/riv/dodaniDat	2012
http://linked.open...aciTvurceVysledku	Bojar, Ondřej Berka, Jan Černý, Martin
http://linked.open.../riv/druhVysledku	J - Článek v odborném periodiku
http://linked.open...iv/duvernostUdaju	S - Úplné a pravdivé údaje nepodléhající ochraně podle zvláštních právních předpisů
http://linked.open...titaPredkladatele	Univerzita Karlova v Praze / Matematicko-fyzikální fakulta
http://linked.open...dnocenehoVysledku	225499
http://linked.open...ai/riv/idVysledku	RIV/00216208:11320/11:10107804
http://linked.open...riv/jazykVysledku	eng - angličtina
http://linked.open.../riv/klicovaSlova	translation; machine; evaluation; based; quiz (en)
http://linked.open.../riv/klicoveSlovo	evaluation machine translation based quiz
http://linked.open...odStatuVydavatele	CZ - Česká republika
http://linked.open...ontrolniKodProRIV	[C003CEEE83C9]
http://linked.open...i/riv/nazevZdroje	The Prague Bulletin of Mathematical Linguistics
http://linked.open...in/vavai/riv/obor	AI
http://linked.open...ichTvurcuVysledku	3 (xsd:int)
http://linked.open...cetTvurcuVysledku	3 (xsd:int)
http://linked.open...vavai/riv/projekt	EuroMatrixPlus—Bringing Machine Translation for European Languages to the User Czech in the Machine Translation Era (CZECHMATE) Combining Phrase-Based and Deep-Syntactic Machine Translation
http://linked.open...UplatneniVysledku	2011
http://linked.open...v/svazekPeriodika	95
http://linked.open...iv/tvurceVysledku	Berka, Jan Bojar, Ondřej Černý, Martin
issn	0032-6585
number of pages	10 (xsd:int)
http://bibframe.org/vocab/doi	10.2478/v10108-011-0006-1
http://localhost/t...ganizacniJednotka	11320

Faceted Search & Find service v1.16.118 as of Jun 21 2024

Alternative Linked Data Documents: ODE Content Formats:

RDF

ODATA

Microdata

About

OpenLink Virtuoso version 07.20.3240 as of Jun 21 2024, on Linux (x86_64-pc-linux-gnu), Single-Server Edition (126 GB total memory, 77 GB memory in use)
Data on this page belongs to its respective rights holders.
Virtuoso Faceted Browser Copyright © 2009-2024 OpenLink Software