AAAI-2022 Hierarchical Heterogeneous Graph Attention Network for Syntax-Aware Summarization

Paper: Hierarchical Heterogeneous Graph Attention Network for Syntax-Aware Summarization

SynapSum: Syntax Graph Attention Network

Abstract

The authors propose a hierarchical heterogeneous graph attention network built on constituency parse trees for syntax-aware summarization, motivated by psychological evidence that people construct summaries hierarchically by selecting specific constituency patterns. Experiments on six benchmark datasets across diverse domains show that the approach is effective for both abstractive and extractive summarization.

Introduction

Summaries produced by existing graph-based methods often diverge semantically from the source text, because the graphs in these models are largely statistical and tend to neglect higher-level semantic information.

Prior work indicates that syntactic structure helps generate compressed yet informative summaries: its hierarchical organization supports pruning less important material and focusing on more salient content, in line with how humans summarize by progressively extracting the most important information to integrate meaning. Syntactic graphs are also typically easier to obtain than semantic graphs, which mitigates the computational cost of earlier methods that construct complex semantic graphs.

Reasons for using constituency parse trees:

• Syntactic dependencies among tokens explicitly encode relational paths.
• Sub-phrases can be extracted straightforwardly from the tree.

Main contributions:

• A heterogeneous graph attention network for syntax-aware summarization on constituency trees.
• State-of-the-art results on six datasets.
• Extension to code summarization by replacing the parse tree with an abstract syntax tree (AST).

Methodology

Experiment

<HR align=left color=#987cb9 SIZE=1>