Machine Translation Evaluation: a Case study of Croatian-English and Russian-English MT Systems

From the earliest days of machine translation (MT), evaluation has been an inherent and significant part of efforts invested into machine translation research. In this paper, an overview of the history of MT evaluation is presented, with emphasis on one of the most comprehensive MT evaluation projec...

Full description

Permalink: http://skupni.nsk.hr/Record/ffzg.KOHA-OAI-FFZG:315091/Details
Glavni autor: Simeon, Ivana (-)
Vrsta građe: Članak
Jezik: eng
Online pristup: http://www.iiassvietri.it/school2006/index_2006.htm
LEADER 02523naa a2200229uu 4500
008 131111s2006 xx 1 eng|d
035 |a (CROSBI)276231 
040 |a HR-ZaFF  |b hrv  |c HR-ZaFF  |e ppiak 
100 1 |a Simeon, Ivana 
245 1 0 |a Machine Translation Evaluation: a Case study of Croatian-English and Russian-English MT Systems /  |c Simeon, Ivana. 
246 3 |i Naslov na engleskom:  |a Machine Translation Evaluation: a Case study of Croatian-English and Russian-English MT Systems 
300 |f str. 
520 |a From the earliest days of machine translation (MT), evaluation has been an inherent and significant part of efforts invested into machine translation research. In this paper, an overview of the history of MT evaluation is presented, with emphasis on one of the most comprehensive MT evaluation projects, undertaken in the 1960s, namely the Automatic Language Processing Advisory Committee Report. Furthermore, strategies and problems pertaining to MT evaluation are discussed, with emphasis on the distinction between subjective criteria, such as comprehensibility, and the objective, quantifiable criteria, such as error quantification and analysis. Within the practical part of the paper, the results of testing four MT systems – one for the language pair Croatian-English, and three for the language pair Russian-English – are shown. The systems were tested on three textual samples belonging to general, fictional and scientific genres. The analysis of the results included a comprehensibility poll which included five informants (native or proficient target language speakers) for each target language, as well as quantification of errors and error type assessment across genres and across individual MT systems. Finally, cumulative results are given for each MT system and for each language pair. As a conclusion, recent developments in the field of MT evaluation are presented, including automatic evaluation methods, such as IBM’ s measures BLEU and NIST. 
536 |a Projekt MZOS  |f 0130418 
546 |a ENG 
690 |a 6.03 
693 |a machine translation, MT evaluation, evaluation criteria  |l hrv  |2 crosbi 
693 |a machine translation, MT evaluation, evaluation criteria  |l eng  |2 crosbi 
773 0 |a The Fundamentals of Verbal and Non-verbal Communication and the Biometrical Issue (2. - 9. rujna 2006. ; Vietri sul Mare) 
856 |u http://www.iiassvietri.it/school2006/index_2006.htm 
942 |c RZB  |u 1  |v Recenzija  |z Znanstveni - Predavanje - Sazetak 
999 |c 315091  |d 315089