ICCV-2019 SlowFast Networks for Video Recognition
SlowFast: Slow and Fast Pathways for Video Recognition
Abstract
The motivation draws on the human visual system, which has two cell types: P-cells (~80%) process static imagery, while M-cells process motion—reminiscent of two-stream designs. The authors propose a network with one slow pathway and one fast pathway.
Method
Slow pathway: Use a very low frame rate (e.g., sample one frame every 16 frames) to learn static appearance and scene context. Because P-cells dominate in number and scene modeling is relatively difficult, most of the model capacity is allocated to the slow branch. In short, the slow pathway is essentially a large I3D-style network; with fewer frames, overall complexity remains moderate.
Fast pathway: Sample more densely (e.g., one frame every 4 frames) and feed the fast branch. This pathway is kept as small as possible to encode motion.
Lateral connections: The two pathways are fused so they interact and jointly learn spatiotemporal features.
The full forward pass and network structure are summarized above.
Experiment
SlowFast accuracy keeps improving as the number of input frames increases and non-local blocks are added.
<HR align=left color=#987cb9 SIZE=1>