OLAC Record oai:lindat.mff.cuni.cz:11372/LRT-2805

Metadata
Title:		WMT18 Quality Estimation Shared Task Test Data
Bibliographic Citation:		http://hdl.handle.net/11372/LRT-2805
Creator:		Specia, Lucia
		Logacheva, Varvara
		Blain, Frederic
		Fernandez, Ramon
		Martins, André
Date (W3CDTF):		2018-05-21T15:23:41Z
Date Available:		2018-05-21T15:23:41Z
Description:		Test data for the WMT18 QE task. Train data can be downloaded from http://hdl.handle.net/11372/LRT-2619. This shared task will build on its previous six editions to further examine automatic methods for estimating the quality of machine translation output at run-time, without relying on reference translations. We include word-level, phrase-level and sentence-level estimation. All tasks make use of datasets produced from post-editions by professional translators. The datasets are domain-specific (IT and life sciences/pharma domains) and extend from those used previous years with more instances and more languages. One important addition is that this year we also include datasets with neural MT outputs. In addition to advancing the state of the art at all prediction levels, our specific goals are: To study the performance of quality estimation approaches on the output of neural MT systems. We will do so by providing datasets for two language language pairs where the same source segments are translated by both a statistical phrase-based and a neural MT system. To study the predictability of deleted words, i.e. words that are missing in the MT output. TO do so, for the first time we provide data annotated for such errors at training time. To study the effectiveness of explicitly assigned labels for phrases. We will do so by providing a dataset where each phrase in the output of a phrase-based statistical MT system was annotated by human translators. To study the effect of different language pairs. We will do so by providing datasets created in similar ways for four language language pairs. To investigate the utility of detailed information logged during post-editing. We will do so by providing post-editing time, keystrokes, and actual edits. Measure progress over years at all prediction levels. We will do so by using last year's test set for comparative experiments. In-house statistical and neural MT systems were built to produce translations for all tasks. MT system-dependent information can be made available under request. The data is publicly available but since it has been provided by our industry partners it is subject to specific terms and conditions. However, these have no practical implications on the use of this data for research purposes. Participants are allowed to explore any additional data and resources deemed relevant.
Identifier (URI):		http://hdl.handle.net/11372/LRT-2805
Language:		English
		German
		Czech
		Latvian
Language (ISO639):		eng
		deu
		ces
		lav
Publisher:		University of Sheffield
Replaces (URI):		http://hdl.handle.net/11372/LRT-2135
Rights:		AGREEMENT ON THE USE OF DATA IN QT21
		https://lindat.mff.cuni.cz/repository/xmlui/page/licence-TAUS_QT21
Subject:		machine translation
		quality estimation
		machine learning
Type:		corpus
Type (DCMI):		Text
Type (OLAC):		primary_text
OLAC Info
Archive:		LINDAT/CLARIAH-CZ digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University
Description:		http://www.language-archives.org/archive/lindat.mff.cuni.cz
GetRecord:		OAI-PMH request for OLAC format
GetRecord:		Pre-generated XML file
OAI Info
OaiIdentifier:		oai:lindat.mff.cuni.cz:11372/LRT-2805
DateStamp:		2021-06-29
GetRecord:		OAI-PMH request for simple DC format
Search Info
Citation:		Specia, Lucia; Logacheva, Varvara; Blain, Frederic; Fernandez, Ramon; Martins, André. 2018. University of Sheffield.
Terms:		area_Europe country_CZ country_DE country_GB dcmi_Text iso639_ces iso639_deu iso639_eng iso639_lav olac_primary_text