Title: Learning Hierarchical Transformers for Scalable and Efficient Text Summarization with Controlled Lengths in Scientific Literature
Authors: Zheng Wang, Xiaoyu Wu, Yuanwen Huang, Jian Sun, Yuqi Liang
Organizations: Microsoft Research Asia; University of Science and Technology Beijing; National Institute for Mathematical Sciences (INICS), Japan; Microsoft Corporation
Abstract: Summarizing scientific literature is a challenging task due to its complex sentence structures and domain-specific terminology. Existing methods often suffer from high computational costs, limiting their scalability in real applications. In this paper, we propose Hierarchical Transformer for Scalable Efficient Text Summarization (HiTESS), which leverages hierarchical attention mechanisms to capture multi-level semantic information and improve summarization quality while maintaining efficiency. Our approach consists of three main components: 1) a document encoder that extracts global contextual features; 2) an abstractive sentence generator with controlled lengths for producing summary sentences; and 3) a postprocessor to refine the output by removing redundancy and improving coherence. We evaluate HiTESS on two benchmark datasets, including SciSumm-English and BioSum, demonstrating significant improvements over state-of-the-art models in both automatic metrics (ROUGE) and human evaluation scores. Furthermore, we showcase the scalability of our method by summarizing thousands of scientific articles within seconds using a single GPU card.
Categories: Computational Linguistics; Natural Language Processing
Keywords: abstractive text summarization; hierarchical attention networks; neural network models; natural language processing; scientific literature analysis
Subjects: Computer Science – Machine Learning (cs.ML); Information Systems – Knowledge Management (cs.IR); Artificial Intelligence
Complete Article after the Jump: Here!