Eye Tracking as an Automatic MT Evaluation Technique

OPEN ARCHIVE

Union Jack
Dannebrog

Eye Tracking as an Automatic MT Evaluation Technique

Show simple item record

dc.contributor.author Carl, Michael
dc.contributor.author Doherty, Stephen
dc.contributor.author O’Brien, Sharon
dc.date.accessioned 2010-04-15
dc.date.accessioned 2010-04-15T11:15:31Z
dc.date.available 2010-04-15T11:15:31Z
dc.date.issued 2010-04-15T11:15:31Z
dc.identifier.uri http://hdl.handle.net/10398/8045
dc.description.abstract Eye tracking has been used successfully as a technique for measuring cognitive load in reading, psycholinguistics, writing, language acquisition etc for some time now. Its application as a technique for automatically measuring the reading ease of MT output has not yet, to our knowledge, been tested. We report here on a preliminary study testing the use and validity of an eye tracking methodology as a means of semi- and/or automatically evaluating machine translation output. 50 French machine translated sentences, 25 rated as excellent and 25 rated as poor in an earlier human evaluation, were selected. 10 native speakers of French were instructed to read the MT sentences for comprehensibility. Their eye gaze data were recorded non-invasively using a Tobii 1750 eye tracker. The average gaze time and fixation count were found to be higher for the “bad” sentences, while average fixation duration and pupil dilations were not found to be substantially different between output rated as good or bad. Comparisons between BLEU scores and eye gaze data were also made and found to correlate well with gaze time and fixation count, and to a lesser extent with pupil dilation and fixation duration. We conclude that the eye tracking data, in particular gaze time and fixation count, correlate reasonably well with human evaluation of MT output but fixation duration and pupil dilation may be less reliable indicators of reading difficulty for MT output. We also conclude that eye tracking has promise as an automatic MT Evaluation technique. en_US
dc.format.extent 13 en_US
dc.language eng en_US
dc.title Eye Tracking as an Automatic MT Evaluation Technique en_US
dc.type art en_US
dc.accessionstatus modt10apr15 liga en_US
dc.contributor.corporation Copenhagen Business School. CBS en_US
dc.contributor.department Institut for Internationale Sprogstudier og Vidensteknologi ( en_US
dc.contributor.departmentshort ISV( en_US
dc.contributor.departmentuk Department of International Language Studies and Computational Linguistics( en_US
dc.contributor.departmentukshort ISV( en_US
dc.idnumber x656526237 en_US
dc.publisher.city Preprint en_US
dc.publisher.year 2010 en_US


Creative Commons License This work is licensed under a Creative Commons License.

Files Size Format View
SubmissionforMT_dohertyobriencarl.pdf 220.9Kb PDF View/Open Preprint version

This item appears in the following Collection(s)

Show simple item record