TransEvalnia: A Prompting-Based System for Fine-Grained, Human-Aligned Translation Evaluation Using LLMs

Recent advancements in large language models (LLMs) have significantly enhanced machine translation capabilities, often surpassing human performance in complex tasks like document-level and literary translation. However, evaluating these high-quality translations remains challenging, as traditional metrics such as BLEU are insufficient for capturing nuanced aspects of translation quality and providing transparent, human-aligned assessments. To address this, the development of systems like TransEvalnia leverages prompting-based techniques with LLMs such as GPT and PaLM2 to deliver fine-grained, explainable evaluations across key dimensions like accuracy, terminology, and audience suitability. These models can perform