ACL-2021 Learning Sequential and Structural Information for Source Code Summarization

Paper: Learning Sequential and Structural Information for Source Code Summarization

mAST + GCN for structure, Transformer for sequence

Abstract

Structural information is encoded with mAST and a GCN; the resulting sequence of AST nodes is then encoded by a Transformer.

Introduction

To represent structural information more effectively, the authors modify the AST by adding sibling edges among nodes at the same level to capture adjacent blocks.

Model

Representing Code as mAST

For Java code, adjacent edges are added between same-level neighbors.

For Python, the authors find function names especially important and therefore introduce a dedicated function-name node with additional edges. #### Proposed Model

The authors argue that mAST captures information from adjacent blocks, GCN brings representations of structurally neighboring nodes closer in semantic space, and the Transformer models long-range dependencies within the same block. ### Experiment

Ablation results and datasets are shown above.

The authors also study where to place the GCN relative to the Transformer encoder (before, after, or both). Performance differs little across placements; placing GCN before the encoder appears slightly preferable.

Experiments on the number of GCN layers are also reported. <HR align=left color=#987cb9 SIZE=1>