Comparison of Multilingual Model Sensitivity for Political Fact Verification with Integrated Multi-Evidence

Abstract

Political news is frequently targeted by the dissemination of fake news on social media, which can influence public opinion and undermine trust in democratic processes. The main challenge in addressing this issue lies in the limited sensitivity of cross-lingual fact verification models in capturing semantic relationships between claims and evidence in long-text, multi-evidence settings. Existing approaches often struggle to assess the relevance and quality of evidence, resulting in suboptimal verification performance. This study compares three multilingual Large Language Models (LLMs), namely mBERT, XLM-R, and LaBSE, for political fact verification using an integrated multi-evidence approach. Experiments are conducted on the PolitiFact dataset, with performance evaluated using sensitivity, accuracy, precision, and F1-score metrics.The results indicate that mBERT achieves the highest overall sensitivity at 89.44%, followed by LaBSE at 81.81% and XLM-R at 78.81%. However, mBERT exhibits lower precision, whereas LaBSE provides a better balance between precision (87.02%) and accuracy (86.46%), resulting in an F1-score of 84.33%. XLM-R demonstrates lower sensitivity but maintains competitive precision (85.47%) and accuracy (84.60%), with an F1-score of 82.00%. Sensitivity analysis based on the number of evidence reveals distinct model behaviors, where mBERT performs optimally with six pieces of evidence, XLM-R is more effective under limited evidence conditions, and LaBSE shows a stable and increasing sensitivity trend as the amount of evidence increases, indicating robustness in multi-evidence scenarios. Further statistical analysis shows that XLM-R has the lowest performance variance, while LaBSE statistically outperforms mBERT in several evaluation aspects. Overall, LaBSE is recommended as the most balanced model for multi-evidence-based political fact verification.