NER model
NER is one of several formulas for accessing live subtitles in television broadcasts and events that are produced using speech recognition. The three letters stand for number, edit error and recognition error. It has been promoted as an alternative to Word error rate (Word Error Rate) which is a more objective measure.
The overall score is calculated as follows: Firstly, the number of edit and recognition errors is deducted from the total number of words in the live subtitles. This number is then divided by the total number of words in the live subtitles and finally multiplied by one hundred. [1]
- .
The acronyms stand for the following:
- N (number) = total number of words in the live subtitles
- E (Edit error) = edit error
- R (Recognition error) = recognition error
This measurement process has been used for public television broadcasts in European countries like Italy and Switzerland.
One major drawback with NER is that it requires a human assessor to rate errors as either:
- 1 Minor edition or recognition errors
- 2 Normal edition or recognition errors
- 3 Serious errors
which are then weighted in the assessment process[2]. This is both subjective, time consuming and costly. Also, NER fails to account for words left out subtitles which is something that does not take account of the D/deaf audience who want verbatim subtitles. As a result, NER cannot accurately reflect the audience's experience of subtitles.
Another problem is the inconsistency of human evaluation of subtitles, particularly with live subtitles, where there are differing opinions of the importance of subtitle errors.[3]
By way of contrast, Word error rate is an objective measure of subtitle errors, since it measures the textual discrepancy between the subtitles and the speech.
See also
[edit]References
[edit]- ^ Romero-Fresco, Pablo: Subtitling through Speech Recognition: Respeaking. Manchester: St. Jerome 2011
- ^ Presentation of the NER model at the International Telecommunication Union – Geneva, November 25, 2011
- ^ Romero-Fresco, Pablo, Óscar Alonso Amigo, and Luis Alonso Bacigalupe: The use of artificial intelligence in the assessment of live subtitling quality: the NER Buddy. Tradumàtica: traducció i tecnologies de la informació i la comunicació 22 (2024): 450-470.