How can I determine if the AI translation is accurate? Does Shangyi AI provide a Quality Estimation (QE) score?
“When translating thousands of files in bulk, users are unable to review each one individually. They need a mechanism to quickly identify which files may have encountered issues.”
Root Cause Analysis
Automated scoring based on COMET/Kiwi
After generating translations, Shangyi AI runs a lightweight Quality Estimation (QE) model. This model does not rely on reference translations; instead, it directly assigns scores based on the semantic alignment between the source and translated text. Segments with low scores will be flagged as 'high risk.'
Uncertainty Detection
LLMs generate Logprobs (logarithmic probabilities) during output. If the AI demonstrates low confidence in every word of a sentence, the system will identify the sentence as potentially containing hallucinations or being difficult to translate. It will then highlight the sentence in the editor to prompt focused human review.
Format Consistency Check
The system also automatically checks for technical errors. For example, if the source text contains three numbers, but the translation contains only two, or if a question mark at the end of the source is missing from the translation. These hard criteria are also included in the quality scoring.
Final Solution Summary
Quantify translation quality with data to enable focused post-editing (PE) and significantly improved efficiency.