Pause Metrics and Machine Translation Utility

OPEN ARCHIVE

Union Jack
Dannebrog

Pause Metrics and Machine Translation Utility

Vis flere oplysninger

Titel: Pause Metrics and Machine Translation Utility
Forfatter: Lacruz, Isabel; Carl, Michael; Yamada, Masaru; Aizawa, Akiko
Resume: Traditionally, attempts to measure Machine Translation (MT) quality have focused on how close output is to a “gold standard” translation. TER (Translation Error Rate) is one standard measure that can be generated automatically. It is the normalized length of the shortest path (smallest number of edits per word) needed to convert MT output to an average of “ideal” translations (Snover et al., 2006). MT quality has now improved so much that post-edited (or in some cases, raw) MT output is routinely used in many applications in place of from-scratch translations. Despite the translators’ continued resistance to post-editing, there is increasing evidence that productivity is greater when translators post-edit rather than translate from scratch (e.g., Green et al., 2013). Machine-assisted alternatives to post-editing, such as Interactive Translation Prediction (see for example Sanchis- Trilles et al., 2014) are also making rapid advances. Because of these changing paradigms, alternative ways of measuring MT quality are being developed. Under many circumstances, perfect accuracy is not necessary: it is enough for MT output to be “good enough.” The end-user of the raw product should be able to use it with little effort, and the posteditor should easily be able to produce a satisfactory product. MT utility is determined by the effect the MT output has on the actual effort expended by the user, while MT adequacy is determined by the anticipated demand the MT output places on the user. Adequacy has been measured by human judgments along Likert scales, as well as by automatic metrics such as TER. In the context of post-editing, TER is modified to HTER, to measure the discrepancy between MT output and the final post-edited product. Thus, HTER measures the smallest number of necessary edits per word during post-editing.
URI: http://hdl.handle.net/10398/9279
Dato: 2016-02-25
Note: Paper presented at The 22nd Annual Meeting of the Association for Natural Language Processing (NLP2016). Tohoku University, Japan, March 2016

Creative Commons License This work is licensed under a Creative Commons License.

Filer Størrelse Format Vis
Michael Cral_2016_01.pdf 456.9Kb PDF Vis/Åbn Conference paper

Dette dokument findes i følgende samling(er)

Vis flere oplysninger