OLAC Record oai:catalogue.elra.info:ELRA-E0024

Metadata
Title:		MEDIA Evaluation Package
Access Rights:		Rights available for: nonCommercialUse, evaluationUse, commercialUse
Date Available (W3CDTF):		2007-06-28
Date Issued (W3CDTF):		2007-06-28
Date Modified (W3CDTF):		2007-06-28
Description:		The MEDIA Evaluation Package was produced within the French national project MEDIA (Automatic evaluation of man-machine dialogue systems), as part of the Technolangue programme funded by the French Ministry of Research and New Technologies (MRNT). The MEDIA project enabled to carry out a campaign for the evaluation of man-machine dialogue systems for French. This package includes the material that was used for the MEDIA evaluation campaign. It includes resources, protocols, scoring tools, results of the campaign, etc., that were used or produced during the campaign. The aim of these evaluation packages is to enable external players to evaluate their own system and compare their results with those obtained during the campaign itself. The campaign is distributed over two actions: 1)Evaluation taking into account the dialogue context: it consists in producing semantic annotation outside dialogue context for each of the 3,000 test prompts.2)Evaluation not taking into account the dialogue context: it consists in evaluating the capacity of understanding systems a) from orthographic transcriptions only and b) from transcriptions and reference annotations outside dialogue context.The MEDIA evaluation package contains the following data and tools:1)Corpus of 1,258 dialogues (WoZ) recorded for a tourist information task, with transcriptions and annotations in dialogue acts and in semantic segments within or outside dialogue context. The annotations follow the XML formalism. This corpus consists of 18,831 user prompts used for the adaptation of systems and for evaluation.2)The text corpus (200 dialogues) is annotated with meta-annotations in order to proceed with a diagnosis study of the systems output. These meta-annotations consist of annotations of speech-oriented phenomena such as repetitions, auto-corrections, incises, etc.3)The Semantizer annotation tool.4)The Mediaval HC (outside context) evaluation tool.5)The Mediaval EC (within context) evaluation tool. A description of the project is available at the following address:http://www.technolangue.net/article.php3?id_article=62 (in French language)
Identifier:		ELRA-E0024
		ISLRN: 699-856-029-354-6
Identifier (URI):		https://catalog.elra.info/en-us/repository/browse/ELRA-E0024/
Language:		French
Language (ISO639):		fra
Medium:		Not specified
Publisher:		ELRA (European Language Resources Association)
Type (DCMI):		Sound
Type (OLAC):		primary_text
OLAC Info
Archive:		ELRA Catalogue of Language Resources
Description:		http://www.language-archives.org/archive/catalogue.elra.info
GetRecord:		OAI-PMH request for OLAC format
GetRecord:		Pre-generated XML file
OAI Info
OaiIdentifier:		oai:catalogue.elra.info:ELRA-E0024
DateStamp:		2007-06-28
GetRecord:		OAI-PMH request for simple DC format
Search Info
Citation:		n.a. 2007. ELRA (European Language Resources Association).
Terms:		area_Europe country_FR dcmi_Sound iso639_fra olac_primary_text