BLEU Evaluation of Machine-Translated English-Croatian Legislation
This paper presents work on the evaluation of online available machine translation (MT) service, i.e. Google Translate, for English-Croatian language pair in the domain of legislation. The total set of 200 sentences, for which three reference translations are provided, is divided into short and long...
| Permalink: | http://skupni.nsk.hr/Record/ffzg.KOHA-OAI-FFZG:317874/Details |
|---|---|
| Matična publikacija: |
Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12) Istanbul, Turkey : European Language Resources Association (ELRA), 2012 |
| Glavni autori: | Seljan, Sanja (-), Vičić, Tomislav (Author), Brkić, Marija |
| Vrsta građe: | Članak |
| Jezik: | eng |
| Online pristup: |
http://www.lrec-conf.org/proceedings/lrec2012/index.html |
| LEADER | 02650naa a2200253uu 4500 | ||
|---|---|---|---|
| 008 | 131111s2012 xx 1 eng|d | ||
| 035 | |a (CROSBI)582714 | ||
| 040 | |a HR-ZaFF |b hrv |c HR-ZaFF |e ppiak | ||
| 100 | 1 | |9 430 |a Seljan, Sanja | |
| 245 | 1 | 0 | |a BLEU Evaluation of Machine-Translated English-Croatian Legislation / |c Seljan, Sanja ; Vičić, Tomislav ; Brkić, Marija. |
| 246 | 3 | |i Naslov na engleskom: |a BLEU Evaluation of Machine-Translated English-Croatian Legislation | |
| 300 | |f str. | ||
| 520 | |a This paper presents work on the evaluation of online available machine translation (MT) service, i.e. Google Translate, for English-Croatian language pair in the domain of legislation. The total set of 200 sentences, for which three reference translations are provided, is divided into short and long sentences. Human evaluation is performed by native speakers, using the criteria of adequacy and fluency. For measuring the reliability of agreement among raters, Fleiss' kappa metric is used. Human evaluation is enriched by error analysis, in order to examine the influence of error types on fluency and adequacy, and to use it in further research. Translation errors are divided into several categories: non-translated words, word omissions, unnecessarily translated words, morphological errors, lexical errors, syntactic errors and incorrect punctuation. The automatic evaluation metric BLEU is calculated with regard to a single and multiple reference translations. System level Pearson’s correlation between BLEU scores based on a single and multiple reference translations is given, as well as correlation between short and long sentences BLEU scores, and correlation between the criteria of fluency and adequacy and each error category. | ||
| 536 | |a Projekt MZOS |f 130-1300646-0909 | ||
| 546 | |a ENG | ||
| 690 | |a 5.04 | ||
| 693 | |a BLEU metric, English-Croatian legislation, human evaluation |l hrv |2 crosbi | ||
| 693 | |a BLEU metric, English-Croatian legislation, human evaluation |l eng |2 crosbi | ||
| 700 | 1 | |a Vičić, Tomislav |4 aut | |
| 700 | 1 | |a Brkić, Marija |4 aut | |
| 773 | 0 | |a Language Resources and Evaluation (LREC'12) (23-25.5.2012. ; LREC 2012) |t Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12) |d Istanbul, Turkey : European Language Resources Association (ELRA), 2012 |n Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis |z 978-2-9517408-7-7 | |
| 856 | |u http://www.lrec-conf.org/proceedings/lrec2012/index.html | ||
| 942 | |c RZB |u 2 |v Recenzija |z Znanstveni - Poster - CijeliRad |t 1.08 | ||
| 999 | |c 317874 |d 317872 | ||