OLAC Record oai:www.ldc.upenn.edu:LDC2007T09

Metadata
Title:		ISI Chinese-English Automatically Extracted Parallel Text
Access Rights:		Licensing Instructions for Subscription & Standard Members, and Non-Members: http://www.ldc.upenn.edu/language-resources/data/obtaining
Bibliographic Citation:		Dragos Stefan Munteanu, and Daniel Marcu. ISI Chinese-English Automatically Extracted Parallel Text LDC2007T09. Web Download. Philadelphia: Linguistic Data Consortium, 2007
Contributor:		Dragos Stefan Munteanu
		Marcu, Daniel
Date (W3CDTF):		2007
Date Issued (W3CDTF):		2007-04-18
Description:		*Introduction* This file contains documentation for ISI Chinese-English Automatically Extracted Parallel Text, Linguistic Data Consortium (LDC) catalog number LDC2007T09 and isbn 1-58563-422-0. This distribution contains a corpus of Chinese-English parallel sentences, which were extracted automatically from two monolingual corpora: Chinese Gigaword Second Edition (LDC2005T14) and English Gigaword Second Edition (LDC2005T12). The data was extracted from news articles published by Xinhua News Agency and was obtained using the automatic parallel sentence identification method described in the following publication: Dragos Stefan Munteanu, Daniel Marcu, 2005. Improving Machine Translation Performance by Exploiting Non-parallel Corpora, Computational Linguistics, 31(4):477-504 The corpus contains 558,567 sentence pairs the word count on the English side is approximately 16M words. The sentences in the parallel corpus preserve the form and encoding of the texts in the original Gigaword corpora. For each sentence pair in the corpus the authors provide the names of the documents from which the two sentences were extracted, as well as a confidence score (between 0.5 and 1.0), which is indicative of their degree of parallelism. The parallel sentence identification approach is designed to judge sentence pairs in isolation from their contexts, and can therefore find parallel sentences within document pairs which are not parallel. The fact that two documents share several parallel sentences does not necessarily mean the documents are parallel In order to make this resource useful for research in Machine Translation (MT), the authors made efforts to detect potential overlaps between this data and the standard test and development data sets used by the MT community. The NIST 2002-2005 MT evaluation data sets contain several articles from Xinhua News Agency. Sentence pairs in this distribution that have a 7-gram overlap with a sentence pair in a NIST MT evaluation set or sentence pairs coming from documents whose names are similar to those in the NIST MT sets are marked with a negative confidence score. *Samples* Please view the following samples: * Chinese Sample * English Sample * Parallel Sample
Extent:		Corpus size: 282624 KB
Identifier:		LDC2007T09
		https://catalog.ldc.upenn.edu/LDC2007T09
		ISBN: 1-58563-422-0
		ISLRN: 224-310-954-973-9
		DOI: 10.35111/6j62-na92
Language:		English
		Mandarin Chinese
Language (ISO639):		eng
		cmn
License:		LDC User Agreement for Non-Members: https://catalog.ldc.upenn.edu/license/ldc-non-members-agreement.pdf
Medium:		Distribution: Web Download
Publisher:		Linguistic Data Consortium
Publisher (URI):		https://www.ldc.upenn.edu
Relation (URI):		https://catalog.ldc.upenn.edu/docs/LDC2007T09
Rights Holder:		Portions © 1990-2004 Xinhua News Agency, © 2005, 2007 Trustees of the University of Pennsylvania
Type (DCMI):		Text
Type (OLAC):		primary_text
OLAC Info
Archive:		The LDC Corpus Catalog
Description:		http://www.language-archives.org/archive/www.ldc.upenn.edu
GetRecord:		OAI-PMH request for OLAC format
GetRecord:		Pre-generated XML file
OAI Info
OaiIdentifier:		oai:www.ldc.upenn.edu:LDC2007T09
DateStamp:		2020-11-30
GetRecord:		OAI-PMH request for simple DC format
Search Info
Citation:		Dragos Stefan Munteanu; Marcu, Daniel. 2007. Linguistic Data Consortium.
Terms:		area_Asia area_Europe country_CN country_GB dcmi_Text iso639_cmn iso639_eng olac_primary_text