Skip to main navigation Skip to search Skip to main content

Improving Factual Consistency in Abstractive Summarization with Sentence Structure Pruning

  • Dingxin Hu
  • , Xuanyu Zhang
  • , Xingyue Zhang
  • , Dongsheng Chen
  • , Yiyang Li
  • , Marina Litvak
  • , Natalia Vanetik
  • , Qing Yang
  • , Dongliang Xu
  • , Yingqi Zhu
  • , Yuze Li
  • , Yanquan Zhou
  • , Lei Li

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

State-of-the-art abstractive summarization models still suffer from the content contradiction between the summaries and the input text, which is referred to as the factual inconsistency problem. Recently, a large number of works have also been proposed to evaluate factual consistency or improve it by post-editing methods. However, these post-editing methods typically focus on replacing suspicious entities, failing to identify and modify incorrect content hidden in sentence structures. In this paper, we first verify that the correctable errors can be enriched by leveraging sentence structure pruning operation, and then we propose a post-editing method based on that. In the correction process, the pruning operation on possible errors is performed on the syntactic dependency tree with the guidance of multiple factual evaluation metrics. Experimenting on the FRANK dataset shows a great improvement in factual consistency compared with strong baselines and, when combined with them, can achieve even better performance. Code and data are availabel at https://github.com/Anthonyhu2333/SSC.

Original languageEnglish
Title of host publication2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings
EditorsNicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
PublisherEuropean Language Resources Association (ELRA)
Pages8792-8803
Number of pages12
ISBN (Electronic)9782493814104
StatePublished - 1 Jan 2024
Externally publishedYes
EventJoint 30th International Conference on Computational Linguistics and 14th International Conference on Language Resources and Evaluation, LREC-COLING 2024 - Hybrid, Torino, Italy
Duration: 20 May 202425 May 2024

Publication series

Name2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings

Conference

ConferenceJoint 30th International Conference on Computational Linguistics and 14th International Conference on Language Resources and Evaluation, LREC-COLING 2024
Country/TerritoryItaly
CityHybrid, Torino
Period20/05/2425/05/24

Keywords

  • abstractive summarization
  • factual inconsistency problem
  • post-editing method

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computational Theory and Mathematics
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Improving Factual Consistency in Abstractive Summarization with Sentence Structure Pruning'. Together they form a unique fingerprint.

Cite this