Pause Metrics and Machine Translation Utility


Union Jack

Pause Metrics and Machine Translation Utility

Show simple item record Lacruz, Isabel Carl, Michael Yamada, Masaru Aizawa, Akiko 2016-02-25T13:24:58Z 2016-02-25T13:24:58Z 2016-02-25
dc.description.abstract Traditionally, attempts to measure Machine Translation (MT) quality have focused on how close output is to a “gold standard” translation. TER (Translation Error Rate) is one standard measure that can be generated automatically. It is the normalized length of the shortest path (smallest number of edits per word) needed to convert MT output to an average of “ideal” translations (Snover et al., 2006). MT quality has now improved so much that post-edited (or in some cases, raw) MT output is routinely used in many applications in place of from-scratch translations. Despite the translators’ continued resistance to post-editing, there is increasing evidence that productivity is greater when translators post-edit rather than translate from scratch (e.g., Green et al., 2013). Machine-assisted alternatives to post-editing, such as Interactive Translation Prediction (see for example Sanchis- Trilles et al., 2014) are also making rapid advances. Because of these changing paradigms, alternative ways of measuring MT quality are being developed. Under many circumstances, perfect accuracy is not necessary: it is enough for MT output to be “good enough.” The end-user of the raw product should be able to use it with little effort, and the posteditor should easily be able to produce a satisfactory product. MT utility is determined by the effect the MT output has on the actual effort expended by the user, while MT adequacy is determined by the anticipated demand the MT output places on the user. Adequacy has been measured by human judgments along Likert scales, as well as by automatic metrics such as TER. In the context of post-editing, TER is modified to HTER, to measure the discrepancy between MT output and the final post-edited product. Thus, HTER measures the smallest number of necessary edits per word during post-editing. en_US
dc.format.extent 4 en_US
dc.language eng en_US
dc.title Pause Metrics and Machine Translation Utility en_US
dc.type cp en_US
dc.contributor.corporation Copenhagen Business School. CBS en_US
dc.contributor.corporation Kent State University, United States en_US
dc.contributor.corporation Kansai University, Japan en_US
dc.contributor.corporation University of Tokyo, National Institute of Informatics, Japan en_US
dc.contributor.department Department of International Business Communication and Politics en_US
dc.contributor.departmentshort IBC en_US
dc.contributor.departmentuk Department of International Business Communication and Politics en_US
dc.contributor.departmentukshort IBC en_US
dc.description.notes Paper presented at The 22nd Annual Meeting of the Association for Natural Language Processing (NLP2016). Tohoku University, Japan, March 2016 en_US Frederiksberg en_US
dc.publisher.year 2016 en_US

Creative Commons License This work is licensed under a Creative Commons License.

Files Size Format View
Michael Cral_2016_01.pdf 456.9Kb PDF View/Open Conference paper

This item appears in the following Collection(s)

Show simple item record